-- Logs begin at Fri 2022-09-09 14:18:42 UTC, end at Fri 2022-09-09 15:13:46 UTC. -- Sep 09 14:42:49.319040 dv-cinder-828696-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 14:42:49.329046 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 14:42:49 2022] *** Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: nodename: dv-cinder-828696-11-iscsi Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: machine: x86_64 Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: clock source: unix Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: pcre jit disabled Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: detected number of CPU cores: 4 Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: current working directory: / Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: detected binary path: /usr/bin/uwsgi-core Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: your processes number limit is 47680 Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: your memory page size is 4096 bytes Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: detected max file descriptor number: 2048 Sep 09 14:42:49.329558 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: lock engine: pthread robust mutexes Sep 09 14:42:49.330191 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: thunder lock: enabled Sep 09 14:42:49.330191 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 14:42:49.330191 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 14:42:49.369192 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: Python main interpreter initialized at 0x5595849a89c0 Sep 09 14:42:49.369192 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: python threads support enabled Sep 09 14:42:49.369192 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: your server socket listen backlog is limited to 100 connections Sep 09 14:42:49.369192 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: your mercy for graceful operations on workers is 90 seconds Sep 09 14:42:49.369569 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 14:42:49.369655 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: *** Operational MODE: preforking *** Sep 09 14:42:49.369898 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: *** uWSGI is running in multiple interpreter mode *** Sep 09 14:42:49.369898 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: spawned uWSGI master process (pid: 81429) Sep 09 14:42:49.370098 dv-cinder-828696-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 14:42:49.370838 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: spawned uWSGI worker 1 (pid: 81443, cores: 1) Sep 09 14:42:49.370973 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: spawned uWSGI worker 2 (pid: 81444, cores: 1) Sep 09 14:42:49.373241 dv-cinder-828696-11-iscsi devstack@keystone.service[81429]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 14:42:53.640543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 14:42:53.640543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: from cryptography.utils import int_from_bytes Sep 09 14:42:53.641350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 14:42:53.641350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: from cryptography.utils import int_from_bytes Sep 09 14:42:53.641765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 14:42:53.641765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: from cryptography.utils import int_from_bytes Sep 09 14:42:53.641878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 14:42:53.641878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: from cryptography.utils import int_from_bytes Sep 09 14:42:53.691433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 14:42:53.691621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 14:42:53.691837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] command line args: [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 14:42:53.692109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 14:42:53.692281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ================================================================================ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 14:42:53.692514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] admin_token = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.692762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config_dir = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.692953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.693137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config_source = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.693441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] debug = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.693599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.695886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.695886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.695886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 14:42:53.696134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 14:42:53.696254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] command line args: [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 14:42:53.696379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_config_append = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 14:42:53.696495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ================================================================================ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 14:42:53.696618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_dir = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] admin_token = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config_dir = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.696977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config_source = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] debug = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.697921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_config_append = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.698044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.698102 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.698162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.698218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_dir = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.699314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.699529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.699529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.699815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.700127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.700127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.700499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.700499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.700876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.700876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.701156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.701497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.701705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] publish_errors = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.701894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.701997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.702317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.702506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.702693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.702945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.703140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.703246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.703430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.703623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.703860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.703951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] standard_threads = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_journal = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_json = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_stderr = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_syslog = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.704951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] publish_errors = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.705071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.705180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.705285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.705353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.705415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.705491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.705610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.705751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.705851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.705952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.706088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] standard_threads = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.706232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.706307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.706435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.706718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_journal = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.706857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_json = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.706990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_stderr = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.706990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_syslog = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.707125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707200 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 14:42:53.707269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.707957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.708884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.709887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.710826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.711972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.712938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.713982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.714782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.714934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.715925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716862 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.716947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.717912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.718712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.718799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.718860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.718916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.718978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.719975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.720918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721585 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.721901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.722864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.723983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.724930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.725995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726863 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.726938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.727874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.728844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.729840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.external = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730700 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.730936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.password = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.token = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.731980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.732944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.external = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.733932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.password = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.token = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.734870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.735982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.736963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.737979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.738970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.739960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740200 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.740940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.741988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.742957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.743992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.744937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.745932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.746892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.747864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.748948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.749997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.750160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.750279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.750406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.750539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.750643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.750796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.750876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.751854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.752967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.753996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.754961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755038 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755200 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.755994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.756966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.757927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.758877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.759988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.760064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.760223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.760300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.760459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.760536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.760696 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.760774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.761485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.761638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.761724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.761794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.761946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.762995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.763948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.764900 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.765862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.766838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.767963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.768930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.769959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.770931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.771975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.772934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.773964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.774929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.775872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.776029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.776105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.776274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.776351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.777927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.778895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.779842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.780970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.781949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.782026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.782183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.782260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.782416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.782501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.782574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 14:42:53.782645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.782878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.783112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.783356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.783592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.783839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.784075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.784321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.784553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.784795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.785030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 14:42:53.785256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 14:42:53.811768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.812001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.812230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.813005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.813262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.813490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.820583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.821747 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.849612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.849852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.850246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.850478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.903837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.903980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.952408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.952544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.952627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:53.952702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.108884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.109104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.109355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.109442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.109576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.109683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.109795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.109878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.110012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.110094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.110246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.110467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.133131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.133390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.133475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.133618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.133722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.133844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.133927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.134066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.134150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.134286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.134370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.134508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.134667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.134747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.146180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.146401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.158756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.158961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.159884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.160018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.160101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.160236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 14:42:54.176707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.176911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 14:42:54.177230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.177316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.177443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 14:42:54.177526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 14:42:54.177732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.177815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.177976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 14:42:54.178057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.178244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.178334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.178506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.178586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.178768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.179031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.190660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.190956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.191234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.191523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.192786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.193075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.194111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.194407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.200919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.203926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.204220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.204510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.206542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.209549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.209836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.210127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.210412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.210593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.210703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.210919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.211001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.211198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.211289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.211478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.211558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.211746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.211824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.212021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.212103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.212293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.212374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.212564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.235570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.235961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.237052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.237364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.237476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.237761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.242137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.242424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.242506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.242710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.242792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.243040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.249731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.250010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.250093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.250297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.255805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.256066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.256149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.256358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.256440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.256640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.256731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.256915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.256996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.257188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.257291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.257483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.257566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.257750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.257833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.258019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.274432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.274662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.274750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.274941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.278661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.278880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.281980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.282213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.282297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.282509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.286570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.286806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.289849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.289985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.293145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.293251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.293469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.293553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.293755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.293838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.294037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.294126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.294317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.294400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.294597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.294680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.310928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.311215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.311298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.311479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.311561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.311738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.311832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.312076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.318684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.319028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.320204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.320588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.321740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.322016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.322162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.322440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.325314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.325598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.325682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.325878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.325961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.326159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.326249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.326483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.326564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.326884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.338171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.338460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.338618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.338743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.338905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.339049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.339214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.339428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.339509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.339718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.339860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.339995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.340157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.340437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.355688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.356230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.358669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.359237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.361612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.362198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.363071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.363718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.364540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.364820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.365217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.365302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.365510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.365910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.372202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.372920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.373699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.374422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.376616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.377011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.377328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.377411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.377615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.377773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.378062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.378346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.385539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.385876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.386903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.387168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.387472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.387762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.393921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.394204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.394483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.394759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.394840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 14:42:54.395046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.395126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.395315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.395404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.395587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.395669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.395860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.395942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.396129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.396219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.396401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.396481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.396677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.396758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.396936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.397016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.397224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.397356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.397805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.397886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.397949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.398064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.398353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 14:42:54.432319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5595849a89c0 pid: 81444 (default app) Sep 09 14:42:54.433828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5595849a89c0 pid: 81443 (default app) Sep 09 14:42:54.436056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538402c9-3a8c-4f9e-9f06-860c28176f05 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:42:54.436339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538402c9-3a8c-4f9e-9f06-860c28176f05 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:42:54.436607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538402c9-3a8c-4f9e-9f06-860c28176f05 None None] PATH_INFO: `/v3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:42:54.440719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Fri Sep 9 14:42:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:01.166640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc70cb-2b0f-4eaf-9303-4088fea26b2c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:01.167141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc70cb-2b0f-4eaf-9303-4088fea26b2c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:01.167248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc70cb-2b0f-4eaf-9303-4088fea26b2c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:01.171441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:01.177006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:01.177573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:01.177573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:01.247439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: INFO dbcounter [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] Registered counter for database keystone Sep 09 14:43:01.270050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG oslo_db.sqlalchemy.engines [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81444) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 14:43:01.276902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writer thread running {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 14:43:01.456379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG passlib.handlers.bcrypt [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81444) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 14:43:01.456756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG passlib.handlers.bcrypt [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81444) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 14:43:01.515040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:01.516633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:01.640632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 09 14:43:02.080084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc70cb-2b0f-4eaf-9303-4088fea26b2c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:02.149547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: INFO dbcounter [None req-ecdc70cb-2b0f-4eaf-9303-4088fea26b2c None None] Registered counter for database keystone Sep 09 14:43:02.170889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG oslo_db.sqlalchemy.engines [None req-ecdc70cb-2b0f-4eaf-9303-4088fea26b2c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81443) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 14:43:02.176937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writer thread running {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 14:43:02.303302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc70cb-2b0f-4eaf-9303-4088fea26b2c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:02.307980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d1eb3-e3ef-47df-997d-6047430219c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:02.308599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d1eb3-e3ef-47df-997d-6047430219c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:02.308874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d1eb3-e3ef-47df-997d-6047430219c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:02.309145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d1eb3-e3ef-47df-997d-6047430219c6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:02.317288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d1eb3-e3ef-47df-997d-6047430219c6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:02.368061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING keystone.common.rbac_enforcer.enforcer [None req-057d1eb3-e3ef-47df-997d-6047430219c6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 14:43:02.368402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d1eb3-e3ef-47df-997d-6047430219c6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:02.378355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Fri Sep 9 14:43:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 14:43:02.384312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:02.402069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28617817-2b23-4b29-b1fa-081c2502f1f4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:02.406471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1f59e-91a1-4cb7-97c9-b2fffd7861ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:02.407575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f59e-91a1-4cb7-97c9-b2fffd7861ef admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:02.407856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f59e-91a1-4cb7-97c9-b2fffd7861ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:02.408129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f59e-91a1-4cb7-97c9-b2fffd7861ef admin admin] PATH_INFO: `/v3/projects/5aff011fa0544761b3d08887135e99aa` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:02.409754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1f59e-91a1-4cb7-97c9-b2fffd7861ef admin admin] RBAC: Authorizing `identity:get_project(project_id=5aff011fa0544761b3d08887135e99aa)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:02.462466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c3a1f59e-91a1-4cb7-97c9-b2fffd7861ef admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 14:43:02.462811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1f59e-91a1-4cb7-97c9-b2fffd7861ef admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:02.467319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Fri Sep 9 14:43:02 2022] GET /identity/v3/projects/5aff011fa0544761b3d08887135e99aa => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:04.573825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c92c5-121e-4386-ac93-aa618948ab1b None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:04.574394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c92c5-121e-4386-ac93-aa618948ab1b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:04.574394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c92c5-121e-4386-ac93-aa618948ab1b None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:04.578470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:04.584118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde3bf-fa13-4b94-a5bb-08aadd8f0e46 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:04.584504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde3bf-fa13-4b94-a5bb-08aadd8f0e46 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:04.584670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde3bf-fa13-4b94-a5bb-08aadd8f0e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:04.604178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d8bde3bf-fa13-4b94-a5bb-08aadd8f0e46 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:04.605691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d8bde3bf-fa13-4b94-a5bb-08aadd8f0e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:04.613838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 14:43:04.877943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c92c5-121e-4386-ac93-aa618948ab1b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:04.890786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c92c5-121e-4386-ac93-aa618948ab1b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:04.894942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddbb17-6d84-46e2-958b-4e46474f718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:04.895537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddbb17-6d84-46e2-958b-4e46474f718b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:04.895816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddbb17-6d84-46e2-958b-4e46474f718b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:04.896089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddbb17-6d84-46e2-958b-4e46474f718b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:04.903596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddbb17-6d84-46e2-958b-4e46474f718b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:04.905379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddbb17-6d84-46e2-958b-4e46474f718b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:04.913945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Fri Sep 9 14:43:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 14:43:04.919501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bde3bf-fa13-4b94-a5bb-08aadd8f0e46 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:04.928515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bde3bf-fa13-4b94-a5bb-08aadd8f0e46 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:04.933385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133d36e-557b-4cc6-9912-43f25204938f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:04.933996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133d36e-557b-4cc6-9912-43f25204938f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:04.934275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133d36e-557b-4cc6-9912-43f25204938f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:04.934543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133d36e-557b-4cc6-9912-43f25204938f admin admin] PATH_INFO: `/v3/users/3ec2e03066934f2597854e78a5c4d345` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:04.935840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133d36e-557b-4cc6-9912-43f25204938f admin admin] RBAC: Authorizing `identity:get_user(user_id=3ec2e03066934f2597854e78a5c4d345)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:04.937688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133d36e-557b-4cc6-9912-43f25204938f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:04.939178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Fri Sep 9 14:43:04 2022] GET /identity/v3/users/3ec2e03066934f2597854e78a5c4d345 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:07.124369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fd306-41a2-4491-a3e3-bb50913b72e1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.124883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fd306-41a2-4491-a3e3-bb50913b72e1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.124883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fd306-41a2-4491-a3e3-bb50913b72e1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.126244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:07.131314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8d6f-f5dc-42f3-81fe-dc2a34397e02 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.131687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8d6f-f5dc-42f3-81fe-dc2a34397e02 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.131868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8d6f-f5dc-42f3-81fe-dc2a34397e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.150457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d7bc8d6f-f5dc-42f3-81fe-dc2a34397e02 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:07.152285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d7bc8d6f-f5dc-42f3-81fe-dc2a34397e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:07.163204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433edc3c-6125-4d43-83df-cd61472327a2 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.163484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433edc3c-6125-4d43-83df-cd61472327a2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.163818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433edc3c-6125-4d43-83df-cd61472327a2 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.165131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:07.170137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.170418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.170684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.212183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG passlib.handlers.bcrypt [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81443) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 14:43:07.212571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG passlib.handlers.bcrypt [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81443) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 14:43:07.233736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:07.262308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:07.263952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:07.363300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:07.500452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8d6f-f5dc-42f3-81fe-dc2a34397e02 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:07.525966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:07.604531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8d6f-f5dc-42f3-81fe-dc2a34397e02 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:07.637859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:07.638188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:07.638498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.638777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.639051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.650037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:07.658902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:07.659982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:07.662245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:07.665619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:07.739045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5af8-879b-40a2-ac85-0d52bb38da85 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:07.741348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:07.746095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:07.766270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d695074-dfe2-4bf6-a717-57d709423a77 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:07.777635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:07.778440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.778716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.778985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.799008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:07.799611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.799888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.800158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.800817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:07.801113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:07.802529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:07.814290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:07.819879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:07.845126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:07.846666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:07.916394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fc052-d70a-4057-a3f8-6d7fe7deb7d8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:07.943842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 14:43:07.949104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:07.949338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:07.949775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:07.950049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:07.950315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] PATH_INFO: `/v3/users/3ec2e03066934f2597854e78a5c4d345` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:07.951447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] RBAC: Authorizing `identity:get_user(user_id=3ec2e03066934f2597854e78a5c4d345)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:07.951959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:07.952994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:07.954468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:07 2022] GET /identity/v3/users/3ec2e03066934f2597854e78a5c4d345 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:07.959655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:07.973935 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:08.048490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6292b4-497a-44ad-a960-bc099107d13f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:08.062017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f039fb-c62a-4800-9d86-e4fe90187169 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:08.081564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:08.082433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:08.082713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:08.082978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:08.093804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:08.094424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:08.094703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:08.094971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:08.096076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:08.097540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:08.099079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:08.100914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:08.102454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:08.105256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:43:08.121552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:08.122444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:43:08.123451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:43:08.123451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:43:08.123451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:08.123451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:43:08.128174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:08.183293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:08.184250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:08.202673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2cb61-5fc5-4a42-ab46-b8c8896e65e1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:08.227631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc9991-c958-48cb-9328-22810480c04f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:08.235562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30cfc8-4fc7-4e4e-bb15-207c5dd0db2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:08.236163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cfc8-4fc7-4e4e-bb15-207c5dd0db2c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:08.236439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cfc8-4fc7-4e4e-bb15-207c5dd0db2c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:08.236706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cfc8-4fc7-4e4e-bb15-207c5dd0db2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:08.237444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30cfc8-4fc7-4e4e-bb15-207c5dd0db2c None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.domain.id=default, user.id=3ec2e03066934f2597854e78a5c4d345) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:08.237745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30cfc8-4fc7-4e4e-bb15-207c5dd0db2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:08.239182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30cfc8-4fc7-4e4e-bb15-207c5dd0db2c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:08.246226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Fri Sep 9 14:43:08 2022] GET /identity/v3/role_assignments?user.id=3ec2e03066934f2597854e78a5c4d345&scope.domain.id=default&role.id=82245735988747a4bc5704727a1cf374 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:08.263029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e9a11-2a6f-4443-bf08-13a922b0b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:08.263661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e9a11-2a6f-4443-bf08-13a922b0b25c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:08.263938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e9a11-2a6f-4443-bf08-13a922b0b25c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:08.264209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e9a11-2a6f-4443-bf08-13a922b0b25c None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:08.264872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e9a11-2a6f-4443-bf08-13a922b0b25c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:08.265172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e9a11-2a6f-4443-bf08-13a922b0b25c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:08.266632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e9a11-2a6f-4443-bf08-13a922b0b25c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:08.286275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:10.738667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9324c-ab0a-43bd-9e1e-fe92aebe0051 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:10.739541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9324c-ab0a-43bd-9e1e-fe92aebe0051 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:10.739966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9324c-ab0a-43bd-9e1e-fe92aebe0051 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:10.741730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:10.746874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d196d-0a2e-4901-8c4f-89da6282b990 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:10.747269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d196d-0a2e-4901-8c4f-89da6282b990 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:10.747433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d196d-0a2e-4901-8c4f-89da6282b990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:10.774459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e34d196d-0a2e-4901-8c4f-89da6282b990 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:10.775734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e34d196d-0a2e-4901-8c4f-89da6282b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:10.874736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:10.935238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531ae39-e8eb-47a0-9eed-d9d252051729 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:10.935565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531ae39-e8eb-47a0-9eed-d9d252051729 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:10.935784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531ae39-e8eb-47a0-9eed-d9d252051729 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:10.937262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:10.942253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5d45a-8ca6-4273-9536-ead33b53e82c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:10.942536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5d45a-8ca6-4273-9536-ead33b53e82c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:10.942804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5d45a-8ca6-4273-9536-ead33b53e82c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:10.944239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 12/25] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:10.945744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f8be-d6ae-4bc7-9571-8e7b5532500d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:10.946025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f8be-d6ae-4bc7-9571-8e7b5532500d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:10.946294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f8be-d6ae-4bc7-9571-8e7b5532500d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:10.952561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d5c3-6d18-4e5b-89b6-a31c9f2a7d3a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:10.954394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d5c3-6d18-4e5b-89b6-a31c9f2a7d3a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:10.954630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d5c3-6d18-4e5b-89b6-a31c9f2a7d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:10.976376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a111f8be-d6ae-4bc7-9571-8e7b5532500d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:10.986603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a111f8be-d6ae-4bc7-9571-8e7b5532500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:10.998978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2358d5c3-6d18-4e5b-89b6-a31c9f2a7d3a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:11.000077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2358d5c3-6d18-4e5b-89b6-a31c9f2a7d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:11.118126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:11.153710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 14/26] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:11.174844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:11.228642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111f8be-d6ae-4bc7-9571-8e7b5532500d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:11.335807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:11.365527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111f8be-d6ae-4bc7-9571-8e7b5532500d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:11.406756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf60a95-48a2-4d3d-8a95-d3ab2935f9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:11.407370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf60a95-48a2-4d3d-8a95-d3ab2935f9a8 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:11.407648 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf60a95-48a2-4d3d-8a95-d3ab2935f9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:11.407920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf60a95-48a2-4d3d-8a95-d3ab2935f9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:11.408595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf60a95-48a2-4d3d-8a95-d3ab2935f9a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:11.410001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf60a95-48a2-4d3d-8a95-d3ab2935f9a8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:11.509575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358d5c3-6d18-4e5b-89b6-a31c9f2a7d3a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:11.564229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 15/28] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 14:43:11 2022] POST /identity/v3/roles => generated 219 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:11.614481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f84a-c820-4566-ac6a-271f65a72d9a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:11.615068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f84a-c820-4566-ac6a-271f65a72d9a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:11.615539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f84a-c820-4566-ac6a-271f65a72d9a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:11.617392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 16/29] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:11.624242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91526-5756-49d6-ad86-e2ce839cc04a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:11.625245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91526-5756-49d6-ad86-e2ce839cc04a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:11.625973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91526-5756-49d6-ad86-e2ce839cc04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:11.675925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fdd91526-5756-49d6-ad86-e2ce839cc04a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:11.677926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fdd91526-5756-49d6-ad86-e2ce839cc04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:11.723809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358d5c3-6d18-4e5b-89b6-a31c9f2a7d3a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:11.782896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:11.783654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:11.783824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:11.784096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:11.808356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:11.816115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:11.817015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:11.818378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:11.818934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 17/30] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:11.818934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91526-5756-49d6-ad86-e2ce839cc04a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:11.829657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 14/31] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:43:11.831421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:11.890510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:11.916280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:11.964281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90469e7-23b8-4ce2-81c7-4397a87446a0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:11.982447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91526-5756-49d6-ad86-e2ce839cc04a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:12.002506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.003787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.004292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.004752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.005639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:12.006145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.007711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.032648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 15/32] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:12.042483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:12.069955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.070609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.071093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.071561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.091344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.093242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:43:12.117699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:12.118561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:43:12.119351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:43:12.119351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:43:12.119351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:12.121585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 18/33] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:12.130406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:12.145669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:12.149588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf16a42-da6e-4904-8fbd-c5f3d115fb5f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:12.186334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.187343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.187857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.188316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] PATH_INFO: `/v3/users/3ec2e03066934f2597854e78a5c4d345` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.190019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] RBAC: Authorizing `identity:get_user(user_id=3ec2e03066934f2597854e78a5c4d345)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.191653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.193702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 16/34] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:12 2022] GET /identity/v3/users/3ec2e03066934f2597854e78a5c4d345 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:12.199596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:12.300356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e241-cdef-49b5-b8e6-17aeb9f9b95e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:12.325587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:12.338927 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921dfd0-ee71-4c7c-908c-f4353300ddfb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:12.360016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.360631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.360911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.361183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.361857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:12.362159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.363557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.383106 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 19/35] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:12.384016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:12.391340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.392130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.392471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.392809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.394175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.395781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.401538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 17/36] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:12.402444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:12.441974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:12.496601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c51c4-58cc-4050-bb49-433ef4d1edb8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:12.512323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ddd07-b872-4de0-9b3c-8a872cb00749 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:12.533182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.533871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.534154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.534502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.535172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.536564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.549726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99af06-2ead-4ac1-9f6f-cdfdddcd6070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.550333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99af06-2ead-4ac1-9f6f-cdfdddcd6070 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.550611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99af06-2ead-4ac1-9f6f-cdfdddcd6070 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.550878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99af06-2ead-4ac1-9f6f-cdfdddcd6070 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.551513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99af06-2ead-4ac1-9f6f-cdfdddcd6070 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.552533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 20/37] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 14:43:12 2022] POST /identity/v3/roles => generated 225 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:12.553175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:12.558059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99af06-2ead-4ac1-9f6f-cdfdddcd6070 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.708305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:12.737702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 18/38] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:43:12 2022] POST /identity/v3/projects => generated 301 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:12.745310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc8c4d-20f8-4ed0-a0db-2b778366ecaa None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.745310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc8c4d-20f8-4ed0-a0db-2b778366ecaa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.745310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc8c4d-20f8-4ed0-a0db-2b778366ecaa None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.745310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 19/39] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:12.749423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06f08-1bd0-4851-b0c6-0b8c49cef1a0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.749713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06f08-1bd0-4851-b0c6-0b8c49cef1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.750017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06f08-1bd0-4851-b0c6-0b8c49cef1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.778755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-14e06f08-1bd0-4851-b0c6-0b8c49cef1a0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:12.780261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-14e06f08-1bd0-4851-b0c6-0b8c49cef1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:12.812585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf96602-6d3d-49cc-94fb-0bf0b814b097 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:12.855422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:12.856209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.856407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.857094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] PATH_INFO: `/v3/domains/default/users/3ec2e03066934f2597854e78a5c4d345/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.858853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3ec2e03066934f2597854e78a5c4d345, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:12.860375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:12.866069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 20/40] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:12.867279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba23a7-a808-4157-8b06-023f4aa3a379 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.867620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba23a7-a808-4157-8b06-023f4aa3a379 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.867936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba23a7-a808-4157-8b06-023f4aa3a379 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.870338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 21/41] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:12.878477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b110b-1d4f-47ee-a80f-d3b2f51f9e38 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:12.879107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b110b-1d4f-47ee-a80f-d3b2f51f9e38 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:12.879539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b110b-1d4f-47ee-a80f-d3b2f51f9e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:12.883067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 21/42] 10.222.0.15 () {58 vars in 1361 bytes} [Fri Sep 9 14:43:12 2022] PUT /identity/v3/domains/default/users/3ec2e03066934f2597854e78a5c4d345/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:12.900616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8e5b110b-1d4f-47ee-a80f-d3b2f51f9e38 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:12.901920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8e5b110b-1d4f-47ee-a80f-d3b2f51f9e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:12.914994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:12.988400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:13.180589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:13.264163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b110b-1d4f-47ee-a80f-d3b2f51f9e38 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:13.276576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:13.284203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69aefe-4787-4b93-bccf-9d7bd9ef2551 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:13.320875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:13.321526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:13.321811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:13.322087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:13.322832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:13.324214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:13.338915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 22/44] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 14:43:13 2022] POST /identity/v3/roles => generated 223 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:13.377515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b110b-1d4f-47ee-a80f-d3b2f51f9e38 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:13.415556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1cbe8-6808-45c9-959e-dc8877673a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:13.416975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1cbe8-6808-45c9-959e-dc8877673a4a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:13.417423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1cbe8-6808-45c9-959e-dc8877673a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:13.417897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1cbe8-6808-45c9-959e-dc8877673a4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:13.419331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1cbe8-6808-45c9-959e-dc8877673a4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:13.421192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1cbe8-6808-45c9-959e-dc8877673a4a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:13.423216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 23/45] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:13.429151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:13.542763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79517cd8-d7de-451e-8f89-fa685a02b986 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:13.576936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6452439-3315-4355-a90f-0a0eabf44cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:13.577594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6452439-3315-4355-a90f-0a0eabf44cd2 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:13.577876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6452439-3315-4355-a90f-0a0eabf44cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:13.578148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6452439-3315-4355-a90f-0a0eabf44cd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:13.578814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6452439-3315-4355-a90f-0a0eabf44cd2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:13.580269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6452439-3315-4355-a90f-0a0eabf44cd2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:13.609417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:43:13 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:43:15.060656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce8078-b48c-46c0-9228-3f102bf7e07a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:15.061171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce8078-b48c-46c0-9228-3f102bf7e07a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:15.061306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce8078-b48c-46c0-9228-3f102bf7e07a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:15.063350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:15.068799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d4e6f-c53c-4c3f-b487-343bfdbcaf69 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:15.069260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d4e6f-c53c-4c3f-b487-343bfdbcaf69 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:15.069400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d4e6f-c53c-4c3f-b487-343bfdbcaf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:15.088044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d72d4e6f-c53c-4c3f-b487-343bfdbcaf69 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:15.089765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d72d4e6f-c53c-4c3f-b487-343bfdbcaf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:15.091794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:15.156416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:15.422448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce8078-b48c-46c0-9228-3f102bf7e07a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:15.441259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:15.530279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce8078-b48c-46c0-9228-3f102bf7e07a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:15.562255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:15.562884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:15.563161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:15.563429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:15.573750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:15.578581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:15.579393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:15.580157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 25/49] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:15.585540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d4e6f-c53c-4c3f-b487-343bfdbcaf69 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:15.686415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d4e6f-c53c-4c3f-b487-343bfdbcaf69 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:15.719467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fb5eb-5e83-4261-bd1b-1f4f9d91e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:15.720072 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fb5eb-5e83-4261-bd1b-1f4f9d91e33b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:15.720349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fb5eb-5e83-4261-bd1b-1f4f9d91e33b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:15.720621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fb5eb-5e83-4261-bd1b-1f4f9d91e33b None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:15.721312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fb5eb-5e83-4261-bd1b-1f4f9d91e33b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:15.721616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fb5eb-5e83-4261-bd1b-1f4f9d91e33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:15.722998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fb5eb-5e83-4261-bd1b-1f4f9d91e33b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:15.727972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:15.733363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 25/50] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:15.737964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:15.750882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bacf1e-faf6-46c6-8479-ccf1247f9f7d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:15.750882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bacf1e-faf6-46c6-8479-ccf1247f9f7d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:15.750882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bacf1e-faf6-46c6-8479-ccf1247f9f7d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:15.750882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 26/51] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:15.754206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3296f02-cc3a-41fd-bf05-153f508b1eed None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:15.754489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3296f02-cc3a-41fd-bf05-153f508b1eed None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:15.754760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3296f02-cc3a-41fd-bf05-153f508b1eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:15.774012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e3296f02-cc3a-41fd-bf05-153f508b1eed None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:15.775201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e3296f02-cc3a-41fd-bf05-153f508b1eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:15.816528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:15.841106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e4872-92ab-4bb7-8aba-ec8f37a25dce None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:15.841822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 27/52] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:15.874182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:15.874809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:15.875088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:15.875387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] PATH_INFO: `/v3/users/3ec2e03066934f2597854e78a5c4d345` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:15.876547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] RBAC: Authorizing `identity:get_user(user_id=3ec2e03066934f2597854e78a5c4d345)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:15.878023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:15.879783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:15 2022] GET /identity/v3/users/3ec2e03066934f2597854e78a5c4d345 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:15.884978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3296f02-cc3a-41fd-bf05-153f508b1eed None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:15.979882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3296f02-cc3a-41fd-bf05-153f508b1eed None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:16.007088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:16.014079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:16.014601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:16.014880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:16.015147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:16.016270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:16.017763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:16.019321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 28/54] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:16.025006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:16.108214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:16.125894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29d89c-6cba-401d-a99e-ac913ccea49a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:16.160460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:16.161055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:16.161374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:16.161655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:16.162356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.domain.id=default, user.id=3ec2e03066934f2597854e78a5c4d345) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:16.162658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:16.164088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:16.172964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 27/55] 10.222.0.15 () {56 vars in 1322 bytes} [Fri Sep 9 14:43:16 2022] GET /identity/v3/role_assignments?user.id=3ec2e03066934f2597854e78a5c4d345&scope.domain.id=default&role.id=82245735988747a4bc5704727a1cf374 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:16.210939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634d612-c340-4227-a66a-93bbbc4661ed None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:16.247222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc51a0a-f278-4c86-b912-f886b91409c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:16.247791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc51a0a-f278-4c86-b912-f886b91409c6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:16.248071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc51a0a-f278-4c86-b912-f886b91409c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:16.248343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc51a0a-f278-4c86-b912-f886b91409c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:16.249527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc51a0a-f278-4c86-b912-f886b91409c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:16.251028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc51a0a-f278-4c86-b912-f886b91409c6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:16.252652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 29/56] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:16.258418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:16.373423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:16.467560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734bd2f-797b-4b3d-9bc1-21834e654370 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:16.499881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a76f8-5690-4be0-9f7c-ceaf935ccbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:16.500421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a76f8-5690-4be0-9f7c-ceaf935ccbf4 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:16.500701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a76f8-5690-4be0-9f7c-ceaf935ccbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:16.500975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a76f8-5690-4be0-9f7c-ceaf935ccbf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:16.501658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a76f8-5690-4be0-9f7c-ceaf935ccbf4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:16.503366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a76f8-5690-4be0-9f7c-ceaf935ccbf4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:16.529504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 28/57] 10.222.0.15 () {60 vars in 1140 bytes} [Fri Sep 9 14:43:16 2022] POST /identity/v3/projects => generated 298 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:18.659034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afacb2e-da0b-4292-8b88-b2bd44ee1629 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:18.659545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afacb2e-da0b-4292-8b88-b2bd44ee1629 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:18.659627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afacb2e-da0b-4292-8b88-b2bd44ee1629 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:18.660905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 30/58] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:18.665992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a133689-f29e-470b-9f14-71760cb869fc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:18.666417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a133689-f29e-470b-9f14-71760cb869fc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:18.666525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a133689-f29e-470b-9f14-71760cb869fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:18.684286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0a133689-f29e-470b-9f14-71760cb869fc None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:18.685497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0a133689-f29e-470b-9f14-71760cb869fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:18.736713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:18.750722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 29/59] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:19.022063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afacb2e-da0b-4292-8b88-b2bd44ee1629 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:19.086250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:19.121431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afacb2e-da0b-4292-8b88-b2bd44ee1629 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:19.157433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf666f46-8ba1-4490-9a94-0ec065ffa049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:19.158050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf666f46-8ba1-4490-9a94-0ec065ffa049 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:19.158351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf666f46-8ba1-4490-9a94-0ec065ffa049 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:19.158624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf666f46-8ba1-4490-9a94-0ec065ffa049 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:19.159724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf666f46-8ba1-4490-9a94-0ec065ffa049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:19.161174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf666f46-8ba1-4490-9a94-0ec065ffa049 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:19.162762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 31/60] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:19.168918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a133689-f29e-470b-9f14-71760cb869fc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:19.262869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a133689-f29e-470b-9f14-71760cb869fc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:19.294630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589e8cc-31d2-4056-855c-8816104afc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:19.295165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589e8cc-31d2-4056-855c-8816104afc77 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:19.295442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589e8cc-31d2-4056-855c-8816104afc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:19.295714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589e8cc-31d2-4056-855c-8816104afc77 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:19.296362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589e8cc-31d2-4056-855c-8816104afc77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:19.297828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589e8cc-31d2-4056-855c-8816104afc77 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:19.322792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 30/61] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:43:19 2022] POST /identity/v3/users => generated 275 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:22.537593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512683f-9acd-4a74-a486-8918f2132f88 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:22.538101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512683f-9acd-4a74-a486-8918f2132f88 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:22.538201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512683f-9acd-4a74-a486-8918f2132f88 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:22.539554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 32/62] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:22.557979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c83c77-5f65-4a24-b15f-8f905f3abc24 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:22.558812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c83c77-5f65-4a24-b15f-8f905f3abc24 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:22.559231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c83c77-5f65-4a24-b15f-8f905f3abc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:22.586224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-78c83c77-5f65-4a24-b15f-8f905f3abc24 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:22.587670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-78c83c77-5f65-4a24-b15f-8f905f3abc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:22.638093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:22.679480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 31/63] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:22.827368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68ff11-ebbb-4ad7-896b-ab66cc3430d2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:22.827863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68ff11-ebbb-4ad7-896b-ab66cc3430d2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:22.828280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68ff11-ebbb-4ad7-896b-ab66cc3430d2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:22.829813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 33/64] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:22.835443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e32ddc-8a6d-4b4c-81d5-3aba495d1d21 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:22.835884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e32ddc-8a6d-4b4c-81d5-3aba495d1d21 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:22.836302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e32ddc-8a6d-4b4c-81d5-3aba495d1d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:22.862744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b1e32ddc-8a6d-4b4c-81d5-3aba495d1d21 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:22.863944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b1e32ddc-8a6d-4b4c-81d5-3aba495d1d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:22.916149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e192d-2283-47ef-a2da-5b76623adc09 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:22.916660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e192d-2283-47ef-a2da-5b76623adc09 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:22.917080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e192d-2283-47ef-a2da-5b76623adc09 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:22.918637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 34/65] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:22.923531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8ab7e-b2ae-4525-908f-b3e9b0d8b1d3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:22.923971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8ab7e-b2ae-4525-908f-b3e9b0d8b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:22.924402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8ab7e-b2ae-4525-908f-b3e9b0d8b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:22.948851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 32/66] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:22.952931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c7b8ab7e-b2ae-4525-908f-b3e9b0d8b1d3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:22.954194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c7b8ab7e-b2ae-4525-908f-b3e9b0d8b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:23.079611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 35/67] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:23.086712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e32ddc-8a6d-4b4c-81d5-3aba495d1d21 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.186725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d7f6-c82e-41b4-a0f2-3011cfba4a1a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.187130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d7f6-c82e-41b4-a0f2-3011cfba4a1a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.187654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d7f6-c82e-41b4-a0f2-3011cfba4a1a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.189370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 36/68] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:23.194200 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecf3dc-65f2-4251-ba29-8dfe59ac3c7c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.194551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecf3dc-65f2-4251-ba29-8dfe59ac3c7c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.195044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecf3dc-65f2-4251-ba29-8dfe59ac3c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.232083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e32ddc-8a6d-4b4c-81d5-3aba495d1d21 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.238241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ceecf3dc-65f2-4251-ba29-8dfe59ac3c7c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:23.239447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ceecf3dc-65f2-4251-ba29-8dfe59ac3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:23.239770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:23.268291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f11bd-475a-4a98-ba6e-2e6018668f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.268928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f11bd-475a-4a98-ba6e-2e6018668f8c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.269218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f11bd-475a-4a98-ba6e-2e6018668f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.269534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f11bd-475a-4a98-ba6e-2e6018668f8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.281636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f11bd-475a-4a98-ba6e-2e6018668f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-cb5f11bd-475a-4a98-ba6e-2e6018668f8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:23.286457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:23.287277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:23.288015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 33/69] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:23.292629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5a3a7-5c82-4033-a4ba-20be1cba0fbf None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.292916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5a3a7-5c82-4033-a4ba-20be1cba0fbf None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.293189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5a3a7-5c82-4033-a4ba-20be1cba0fbf None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.294634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 34/70] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:23.298560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5a3a7-5c82-4033-a4ba-20be1cba0fbf None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.308704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:23.317804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 37/71] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:23.318611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecf3dc-65f2-4251-ba29-8dfe59ac3c7c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.428772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecf3dc-65f2-4251-ba29-8dfe59ac3c7c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.431049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5a3a7-5c82-4033-a4ba-20be1cba0fbf None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.464676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e9645-a007-4ea7-a2c0-95c383f99791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.465539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e9645-a007-4ea7-a2c0-95c383f99791 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.465955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e9645-a007-4ea7-a2c0-95c383f99791 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.466364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e9645-a007-4ea7-a2c0-95c383f99791 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.467157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e9645-a007-4ea7-a2c0-95c383f99791 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:23.467608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e9645-a007-4ea7-a2c0-95c383f99791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:23.467730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.468339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.468620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.468895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.469677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e9645-a007-4ea7-a2c0-95c383f99791 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:23.477826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 38/72] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:23.479165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00596-0be3-4b17-a870-af6c2097317c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.479443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00596-0be3-4b17-a870-af6c2097317c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.479716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00596-0be3-4b17-a870-af6c2097317c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.482451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:23.487765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:23.488580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:23.489441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 35/73] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:23.490047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.505463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7fe00596-0be3-4b17-a870-af6c2097317c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:23.506860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7fe00596-0be3-4b17-a870-af6c2097317c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:23.517621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:23.566724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:23.587822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 39/74] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:23.588923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00596-0be3-4b17-a870-af6c2097317c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.600557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef032df7-417a-4b57-b6db-cba800ae20e1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.636987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.637677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.637963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.638238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.649590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:23.654152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:23.654996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:23.655749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 36/75] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:23.656365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.697247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00596-0be3-4b17-a870-af6c2097317c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.731567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.732198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.732482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.732756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.746725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:23.748189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:23.750014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 40/76] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:23.750922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.754950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:23.774802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4bf5-10a0-4bff-9598-e500d42c1068 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.806217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:23.812376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.812989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.813293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.813572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.814245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:23.814545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:23.815939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:23.824400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 37/77] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:23.825012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.866225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aa12f-4bc7-43c7-85e5-5ccfebf56310 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.899433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.900057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.900336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.900632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.901773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:23.903239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:23.904794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 41/78] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:23.905568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.927052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc7e2f-b06a-461f-8cca-679001890369 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:23.960474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:23.961300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:23.961730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:23.962005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:23.962665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:23.963006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:23.964433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:23.972390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 38/79] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:23.973612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:23.986731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.004204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fac33a-9753-46b9-af7a-b2c63ed4c35e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.016971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.038191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.038812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.039094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.039368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.080482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabb7a3-72b9-4cf3-84ab-9fe1ad21612b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.105884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.107363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.113090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.113757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.114038 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.114312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] PATH_INFO: `/v3/users/3ec2e03066934f2597854e78a5c4d345` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.115458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] RBAC: Authorizing `identity:get_user(user_id=3ec2e03066934f2597854e78a5c4d345)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.116883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.118496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 39/80] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/users/3ec2e03066934f2597854e78a5c4d345 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:24.119253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.199217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.210054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 42/81] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:24.210944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.222868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b28a6-c42a-47ce-a056-a1bb84cd85c3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.224924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.255770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.256402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.256682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.256959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.267081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:24.272283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:24.273313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:24.274087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 40/82] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:24.274931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.309667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99673d-76ff-400f-abfc-ba62b7ed93f5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.343046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.343650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.343937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.344214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.344867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.346351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.372489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114b7a-d804-41d6-9dd5-fd711429d63b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.373485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 43/83] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:43:24 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:24.378221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.414024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.415052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.415595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.416008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.417418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.419048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.420673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 41/84] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:24.421475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.425478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.455060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.496111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b9896-48c4-405a-b444-ea70f0881d6a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.533551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.534188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.535032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.535328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.547260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf2028-5ea9-4ef4-84ec-95daeff47246 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.585555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.586208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.586676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.586936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.588667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.590215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.592188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 42/85] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:24.593029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.623725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.627613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.674754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.707247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.712383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f3073-31f4-4a8a-bf91-86a115edbd61 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.749736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 44/86] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:24.750531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.759155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.760883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.761500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.761764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.762890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:24.763209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.765341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.776903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 43/87] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:24.777984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.876536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef189-b37d-43a3-ae7e-697d207b3f3b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.913711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.913908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8a3fb-cab8-4fcd-9b5e-97ff2616a991 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:24.914327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.914670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.914948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.955753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.963308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:24.963988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:24.964782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:24.965068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:24.966262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=55f51cbb4c144d569e357010b10f0ed6) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:24.966769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.968620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:24.971356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:24.978133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/role_assignments?user.id=55f51cbb4c144d569e357010b10f0ed6&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:24.979643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:24.994077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:24.996162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:25.127509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 45/89] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:25.128853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:25.191442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52351f8-4698-4eed-9f8f-65af20dab40a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:25.209331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:25.257672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccb2a4-92ff-4528-9dab-756f96c0951e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:25.285750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:25.286320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:25.286603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:25.286881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:25.288054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:25.292850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:25.295141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 45/90] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:24 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:25.295944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:25.303154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:25.303818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:25.305060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:25.305485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:25.308701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:25.310242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:25.311982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 46/91] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:25 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:25.312770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:25.403649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:25.480509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefefc83-bc2e-41b9-91cb-1234e2cbabf8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:25.526256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:25.526896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:25.527174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:25.527448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:25.528587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:25.530136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:25.531946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 46/92] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:25 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:25.532525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:25.544140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0831d0-3593-4b88-97ba-018e6b4ed7a1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:25.614949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:25.615579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:25.615862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:25.616138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:25.616870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=3ec2e03066934f2597854e78a5c4d345) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:25.617176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:25.618666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:25.626865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:25.632214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 47/93] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:25 2022] GET /identity/v3/role_assignments?user.id=3ec2e03066934f2597854e78a5c4d345&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:25.633120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:25.680311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff6e8-2a93-43fa-9c27-375d2358fd0e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:25.727524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:25.728149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:25.728432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:25.729358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:25.738824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:25.831556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf73de7-cc1c-47d7-bd44-cb2a3bf48a2b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:25.858037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:25.859560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:25.900251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:25.900976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:25.901282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:25.902638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:25.903389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] RBAC: Adding query filter params (role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=55f51cbb4c144d569e357010b10f0ed6) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:25.903706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:25.905192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:25.919100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:25 2022] GET /identity/v3/role_assignments?user.id=55f51cbb4c144d569e357010b10f0ed6&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:26.044796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 47/95] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:26.050568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:26.135562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:26.191469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3de2c-33d8-460c-9368-2f1e4471770e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:26.238172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83722b4e-27ba-4584-a5f9-36d3e7efe446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:26.238820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722b4e-27ba-4584-a5f9-36d3e7efe446 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:26.239103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722b4e-27ba-4584-a5f9-36d3e7efe446 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:26.239378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722b4e-27ba-4584-a5f9-36d3e7efe446 None admin] PATH_INFO: `/v3/projects/273dc016cd07438c81e7d37675d1daa4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:26.240532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83722b4e-27ba-4584-a5f9-36d3e7efe446 None admin] RBAC: Authorizing `identity:get_project(project_id=273dc016cd07438c81e7d37675d1daa4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:26.242018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83722b4e-27ba-4584-a5f9-36d3e7efe446 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:26.243665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 49/96] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:26 2022] GET /identity/v3/projects/273dc016cd07438c81e7d37675d1daa4 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:26.250170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:26.286027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:26.359275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc0e8c-bf0a-490c-8047-a4a634fb11f7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:26.398109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcae73-532d-47c5-a1a0-675d3c6e95cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:26.398702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcae73-532d-47c5-a1a0-675d3c6e95cc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:26.398986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcae73-532d-47c5-a1a0-675d3c6e95cc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:26.399264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcae73-532d-47c5-a1a0-675d3c6e95cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:26.406183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcae73-532d-47c5-a1a0-675d3c6e95cc None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=273dc016cd07438c81e7d37675d1daa4, user.id=55f51cbb4c144d569e357010b10f0ed6) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:26.406514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcae73-532d-47c5-a1a0-675d3c6e95cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:26.408017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcae73-532d-47c5-a1a0-675d3c6e95cc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:26.413807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 48/97] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:26 2022] GET /identity/v3/role_assignments?user.id=55f51cbb4c144d569e357010b10f0ed6&scope.project.id=273dc016cd07438c81e7d37675d1daa4&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:27.286959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa28ef-b81e-47c3-a65a-bf9772c6ee55 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:27.287964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa28ef-b81e-47c3-a65a-bf9772c6ee55 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:27.288465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa28ef-b81e-47c3-a65a-bf9772c6ee55 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:27.290044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 50/98] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:27.295696 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8807-7c50-4f4f-ae11-9b2ef8d33c4c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:27.296501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8807-7c50-4f4f-ae11-9b2ef8d33c4c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:27.296988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8807-7c50-4f4f-ae11-9b2ef8d33c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:27.321649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-35ff8807-7c50-4f4f-ae11-9b2ef8d33c4c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:27.323744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-35ff8807-7c50-4f4f-ae11-9b2ef8d33c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:27.392696 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 49/99] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:27.653438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d272d-adc6-4639-8ba7-64ce7dc3bc67 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:27.654120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d272d-adc6-4639-8ba7-64ce7dc3bc67 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:27.654609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d272d-adc6-4639-8ba7-64ce7dc3bc67 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:27.656178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 51/100] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:27.661512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb793b-ba1d-4e1c-9056-c72250e470ff None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:27.662039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb793b-ba1d-4e1c-9056-c72250e470ff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:27.662526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb793b-ba1d-4e1c-9056-c72250e470ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:27.684030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c5fb793b-ba1d-4e1c-9056-c72250e470ff None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:27.685580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c5fb793b-ba1d-4e1c-9056-c72250e470ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:27.712681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:27.766166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d272d-adc6-4639-8ba7-64ce7dc3bc67 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:27.783377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 50/101] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:27.927385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d272d-adc6-4639-8ba7-64ce7dc3bc67 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:27.965592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:27.970442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7aaf0-4ef8-4f08-af99-c2f787fbaa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:27.971074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7aaf0-4ef8-4f08-af99-c2f787fbaa68 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:27.971354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7aaf0-4ef8-4f08-af99-c2f787fbaa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:27.971632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7aaf0-4ef8-4f08-af99-c2f787fbaa68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:27.972749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7aaf0-4ef8-4f08-af99-c2f787fbaa68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:27.974266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7aaf0-4ef8-4f08-af99-c2f787fbaa68 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:27.976007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 52/102] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:27.981983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb793b-ba1d-4e1c-9056-c72250e470ff None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:28.030844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31db41-b349-4efb-bcf0-d48c2c92e70e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:28.031131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31db41-b349-4efb-bcf0-d48c2c92e70e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:28.031405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31db41-b349-4efb-bcf0-d48c2c92e70e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:28.032747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 53/103] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:28.043275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d8391-1b17-4301-8266-6ed56b54fd41 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:28.043565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d8391-1b17-4301-8266-6ed56b54fd41 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:28.043845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d8391-1b17-4301-8266-6ed56b54fd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:28.080104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a65d8391-1b17-4301-8266-6ed56b54fd41 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:28.081354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a65d8391-1b17-4301-8266-6ed56b54fd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:28.185035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb793b-ba1d-4e1c-9056-c72250e470ff None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:28.228474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:28.229100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:28.229413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:28.229681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:28.230346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:28.231918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:28.239427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 54/104] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:28.285303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 51/105] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:43:27 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:28.413370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d8391-1b17-4301-8266-6ed56b54fd41 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:28.557521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d8391-1b17-4301-8266-6ed56b54fd41 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:28.596103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:28.604084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:28.604744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:28.605024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:28.605325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:28.626233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:28.631177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:28.632035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:28.633282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 55/106] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:28.641224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:28.654534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:28.675431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:28.799661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2767c-3f3d-4c28-bb91-e68f82db2dce None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:28.829259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8628c-08a1-47c0-871f-0cb6a7323f3b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:28.841529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:28.842459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:28.842956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:28.843422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:28.856189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:28.864463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:28.865339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:28.866091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:28.872087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 56/107] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:43:28.879694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:28.905517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:28.906496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:28.907030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:28.907526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:28.908473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:28.910993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:28.912634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:28.927861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 52/108] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:28.939252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:28.995287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:29.066410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:29.068425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9813873-8f5b-4e90-a948-2a01a7722c18 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:29.107203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c84db-47f3-420c-a74d-a660078eac98 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:29.108808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f089073-0858-42e4-a934-12bd87fbde84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:29.109693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f089073-0858-42e4-a934-12bd87fbde84 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.110108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f089073-0858-42e4-a934-12bd87fbde84 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.110515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f089073-0858-42e4-a934-12bd87fbde84 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.111308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f089073-0858-42e4-a934-12bd87fbde84 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:29.111763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f089073-0858-42e4-a934-12bd87fbde84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:29.113335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f089073-0858-42e4-a934-12bd87fbde84 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:29.123187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 57/109] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:29.124568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352596c1-6dbb-431b-b0b9-5061fb7d35ff None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.125069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352596c1-6dbb-431b-b0b9-5061fb7d35ff None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.125522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352596c1-6dbb-431b-b0b9-5061fb7d35ff None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.126862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 58/110] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:29.132158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cb4f9-300d-4dcd-9768-4eed0bd3d016 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.132608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cb4f9-300d-4dcd-9768-4eed0bd3d016 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.133029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cb4f9-300d-4dcd-9768-4eed0bd3d016 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.134860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 59/111] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:29.135758 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cb4f9-300d-4dcd-9768-4eed0bd3d016 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:29.151107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4867e-bdb2-49de-a3cc-e5f498ae4bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:29.151784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4867e-bdb2-49de-a3cc-e5f498ae4bae None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.152416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4867e-bdb2-49de-a3cc-e5f498ae4bae None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.152698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4867e-bdb2-49de-a3cc-e5f498ae4bae None admin] PATH_INFO: `/v3/users/33023b1c1c8f45e1a9cea5b6f7c0f437` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.174182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4867e-bdb2-49de-a3cc-e5f498ae4bae None admin] RBAC: Authorizing `identity:get_user(user_id=33023b1c1c8f45e1a9cea5b6f7c0f437)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:29.176036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4867e-bdb2-49de-a3cc-e5f498ae4bae None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:29.177931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 53/112] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:28 2022] GET /identity/v3/users/33023b1c1c8f45e1a9cea5b6f7c0f437 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:29.183011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d97f4-d4d3-4a7a-a774-6a9f63e685c6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.184239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d97f4-d4d3-4a7a-a774-6a9f63e685c6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.184648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d97f4-d4d3-4a7a-a774-6a9f63e685c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.209697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2d7d97f4-d4d3-4a7a-a774-6a9f63e685c6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:29.211175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2d7d97f4-d4d3-4a7a-a774-6a9f63e685c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:29.257937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cb4f9-300d-4dcd-9768-4eed0bd3d016 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:29.274442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:29.295719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 54/113] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:29.301406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:29.302290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.302640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.302992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.303397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7306e-7959-4ede-99da-5d64bd53eff6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.303833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7306e-7959-4ede-99da-5d64bd53eff6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.304241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7306e-7959-4ede-99da-5d64bd53eff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.304388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:29.305958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:29.310050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 60/114] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:29 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:29.313075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:29.339272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-22e7306e-7959-4ede-99da-5d64bd53eff6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:29.340781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-22e7306e-7959-4ede-99da-5d64bd53eff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:29.431360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d4e9e-8362-4c54-b823-bee19e4aa1cd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:29.447062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:29.468245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 55/115] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:29.469298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7306e-7959-4ede-99da-5d64bd53eff6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:29.473479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2f546-aeb0-46de-816d-b2c453a597ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:29.474251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2f546-aeb0-46de-816d-b2c453a597ec None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.474626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2f546-aeb0-46de-816d-b2c453a597ec None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.474967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2f546-aeb0-46de-816d-b2c453a597ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.531668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:29.557340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2f546-aeb0-46de-816d-b2c453a597ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:29.558983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2f546-aeb0-46de-816d-b2c453a597ec None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:29.681060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 61/116] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:29.682589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df23a8-014a-4424-b19e-9a1658a84638 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.682898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df23a8-014a-4424-b19e-9a1658a84638 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.685640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df23a8-014a-4424-b19e-9a1658a84638 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.685640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 62/117] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:29.686250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df23a8-014a-4424-b19e-9a1658a84638 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:29.737869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7306e-7959-4ede-99da-5d64bd53eff6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:29.789574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:29.798437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df23a8-014a-4424-b19e-9a1658a84638 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:29.836514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:29.837261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.837550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.838442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.839572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:29.841123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:29.842492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72d962-be8d-4bde-9408-8e34ad2ca075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:29.843141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72d962-be8d-4bde-9408-8e34ad2ca075 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.843419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72d962-be8d-4bde-9408-8e34ad2ca075 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.843695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72d962-be8d-4bde-9408-8e34ad2ca075 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.844834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72d962-be8d-4bde-9408-8e34ad2ca075 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:29.846286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72d962-be8d-4bde-9408-8e34ad2ca075 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:29.847951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 56/119] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:29 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:29.848148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 63/119] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:29 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:29.849064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c9ec-c151-4d1f-8dbc-3830675f4d65 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:29.853661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:29.857485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c9ec-c151-4d1f-8dbc-3830675f4d65 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:29.857800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c9ec-c151-4d1f-8dbc-3830675f4d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:29.892547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-09e4c9ec-c151-4d1f-8dbc-3830675f4d65 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:29.893775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-09e4c9ec-c151-4d1f-8dbc-3830675f4d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:29.952308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:29.966880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 57/120] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:29.967510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c9ec-c151-4d1f-8dbc-3830675f4d65 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:29.978691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768df3-ce35-49a8-89b4-6c87e3ccaf16 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.026984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.027664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.027990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.028246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/55f51cbb4c144d569e357010b10f0ed6/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.033629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4eabd47bff3c4b6eb0e462a0797efb23, user_id=55f51cbb4c144d569e357010b10f0ed6, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:30.035085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:30.051374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 64/121] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:29 2022] PUT /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/55f51cbb4c144d569e357010b10f0ed6/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:30.052839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.068204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:30.099096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c9ec-c151-4d1f-8dbc-3830675f4d65 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.147525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.148154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.148442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.148718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.149499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] RBAC: Adding query filter params (role.id=5b6d374901c24586a24d2483221bed64, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=33023b1c1c8f45e1a9cea5b6f7c0f437) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:30.149808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:30.152179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:30.159908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 58/122] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:29 2022] GET /identity/v3/role_assignments?user.id=33023b1c1c8f45e1a9cea5b6f7c0f437&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=5b6d374901c24586a24d2483221bed64 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:30.160786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.192000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d465cf4-3385-4cfc-96e1-cb8e50f4b6d2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.230586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.231597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.232087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.232600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.238026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:30.247080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:30.253084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:30.254003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:30.254729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:30.257138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 65/123] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:30.263721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.302744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a9aa-a233-49d1-ba2f-60bd6506b5b9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.349729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.350545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.350843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.351120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.357581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:30.366332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:30.373089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:30.374105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:30.374836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 59/124] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:30.380315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.409812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ac96-dd6c-4c64-9894-6430b64aa443 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.457479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.458133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.458437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.458721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.459469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:30.459786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:30.461300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:30.475668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 66/125] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:30.476479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.530685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c101d-fc82-4f79-8888-412bf3bfd82f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.547288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:30.593937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.594978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.595464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.595937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.596807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:30.597329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:30.598942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:30.621314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dcad7-6eeb-4b80-8bff-cca938712a32 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.623325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 60/126] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:30.623974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.673573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:30.686769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.687794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.688346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.688813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.710649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:30.718252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:30.719161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:30.719918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:30.724555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 67/127] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:30.730033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.819966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f537eb1-816e-4eb4-b450-053506701408 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.901247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2bb84-88fd-45c6-bd13-c120313ff529 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:30.937232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.938215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.938592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.939050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] PATH_INFO: `/v3/users/3ec2e03066934f2597854e78a5c4d345` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.940357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] RBAC: Authorizing `identity:get_user(user_id=3ec2e03066934f2597854e78a5c4d345)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:30.942144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:30.942956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:30.943824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:30.944017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 68/128] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/users/3ec2e03066934f2597854e78a5c4d345 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:30.944456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:30.944932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:30.945050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:30.948961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:30.950637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:30.956694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 61/129] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:30.957750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:31.009606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:31.067789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844ec81-b064-4a81-aab9-44cda5d244dd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:31.101448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:31.107300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:31.107958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.108328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.109098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.109814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:31.110118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:31.112320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:31.122563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 69/130] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:31.123451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:31.214597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab6cca-edba-464d-b90a-2dde46540ee9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:31.248672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b06fe-63c5-4da3-be42-3eed0bbd0f36 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:31.252669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:31.253338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.253621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.253893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.255043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:31.256581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:31.258285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 62/131] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:30 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:31.259160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:31.273393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:31.293612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f81b3-4019-45c6-9dde-10733c70dd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:31.294690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f81b3-4019-45c6-9dde-10733c70dd6c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.295036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f81b3-4019-45c6-9dde-10733c70dd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.295306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f81b3-4019-45c6-9dde-10733c70dd6c None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.296744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f81b3-4019-45c6-9dde-10733c70dd6c None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:31.298500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f81b3-4019-45c6-9dde-10733c70dd6c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:31.301028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 70/132] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:31 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:31.302079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf78bb-9b96-4b0c-a47c-e840894870f3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.302363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf78bb-9b96-4b0c-a47c-e840894870f3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.302888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf78bb-9b96-4b0c-a47c-e840894870f3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.305123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 71/133] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:31.309152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf78bb-9b96-4b0c-a47c-e840894870f3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:31.388015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8e7d8-9dd6-4b5a-92f5-1fff8fbbb9e4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:31.400979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:31.430838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:31.431808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.432098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.432373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.433881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:31.435836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:31.437878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 63/134] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:31 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:31.438613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:31.443228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf78bb-9b96-4b0c-a47c-e840894870f3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:31.480434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59382b-48f1-431e-83dc-1b2903796fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:31.481999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59382b-48f1-431e-83dc-1b2903796fae None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.482513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59382b-48f1-431e-83dc-1b2903796fae None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.482986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59382b-48f1-431e-83dc-1b2903796fae None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/3ec2e03066934f2597854e78a5c4d345/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.485246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59382b-48f1-431e-83dc-1b2903796fae None admin] RBAC: Authorizing `identity:create_grant(project_id=4eabd47bff3c4b6eb0e462a0797efb23, user_id=3ec2e03066934f2597854e78a5c4d345, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:31.486957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59382b-48f1-431e-83dc-1b2903796fae None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:31.508923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 72/135] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:31 2022] PUT /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/3ec2e03066934f2597854e78a5c4d345/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:31.512567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ee91-6bc5-4e2c-a912-4cf35aac9b6f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.513118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ee91-6bc5-4e2c-a912-4cf35aac9b6f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.513926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ee91-6bc5-4e2c-a912-4cf35aac9b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.539609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0d10ee91-6bc5-4e2c-a912-4cf35aac9b6f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:31.541442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0d10ee91-6bc5-4e2c-a912-4cf35aac9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:31.567463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bf65f-ea7b-4a3d-9d3f-21489f5ce4fd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:31.597733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:31.633778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:31.635905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.636744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.638115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/55f51cbb4c144d569e357010b10f0ed6/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.643707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] RBAC: Authorizing `identity:create_grant(project_id=4eabd47bff3c4b6eb0e462a0797efb23, user_id=55f51cbb4c144d569e357010b10f0ed6, role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:31.646625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:31.658247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 73/136] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:31.660862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10ee91-6bc5-4e2c-a912-4cf35aac9b6f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:31.685859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 64/137] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:31 2022] PUT /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/55f51cbb4c144d569e357010b10f0ed6/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:31.819776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10ee91-6bc5-4e2c-a912-4cf35aac9b6f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:31.864574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:31.865243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:31.865539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:31.865810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] PATH_INFO: `/v3/projects/273dc016cd07438c81e7d37675d1daa4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:31.866967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] RBAC: Authorizing `identity:get_project(project_id=273dc016cd07438c81e7d37675d1daa4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:31.868453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:31.870688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 74/138] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:31 2022] GET /identity/v3/projects/273dc016cd07438c81e7d37675d1daa4 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:31.876790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:31.931469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:32.002582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0b4f-8e87-4a1c-9cf5-166c15b6bb5b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:32.042604 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:32.043582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:32.044059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:32.044533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] PATH_INFO: `/v3/projects/273dc016cd07438c81e7d37675d1daa4/users/55f51cbb4c144d569e357010b10f0ed6/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:32.046726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=273dc016cd07438c81e7d37675d1daa4, user_id=55f51cbb4c144d569e357010b10f0ed6, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:32.048525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:32.073029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 65/139] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:31 2022] PUT /identity/v3/projects/273dc016cd07438c81e7d37675d1daa4/users/55f51cbb4c144d569e357010b10f0ed6/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:32.110470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:32.182625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:32.259366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7dfd5-8ed9-443a-b7ff-a12f75184b7a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:32.304607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cb024-7871-4cd1-9f28-1f791a8494e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:32.305263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cb024-7871-4cd1-9f28-1f791a8494e0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:32.305546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cb024-7871-4cd1-9f28-1f791a8494e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:32.305822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cb024-7871-4cd1-9f28-1f791a8494e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:32.309905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cb024-7871-4cd1-9f28-1f791a8494e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:32.311398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cb024-7871-4cd1-9f28-1f791a8494e0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:32.313337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 75/140] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:32.318878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:32.467395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:32.471877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7030c9-1ebc-45f0-833d-2b93b6d8cfc5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:32.516417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22e9cd-b51d-47fa-b73c-9b3d4b2d7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:32.517236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e9cd-b51d-47fa-b73c-9b3d4b2d7f0c None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:32.517506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e9cd-b51d-47fa-b73c-9b3d4b2d7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:32.517782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e9cd-b51d-47fa-b73c-9b3d4b2d7f0c None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:32.518453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22e9cd-b51d-47fa-b73c-9b3d4b2d7f0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:32.519897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22e9cd-b51d-47fa-b73c-9b3d4b2d7f0c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:32.546640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 66/141] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:43:32 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:32.907336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b675842-0332-4a9e-be3b-1dcdf52ce35e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:32.907845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b675842-0332-4a9e-be3b-1dcdf52ce35e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:32.907917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b675842-0332-4a9e-be3b-1dcdf52ce35e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:32.909525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 76/142] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:32.914809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e725197-ee9f-4b52-85d6-c2b53e6117b1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:32.915214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e725197-ee9f-4b52-85d6-c2b53e6117b1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:32.915443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e725197-ee9f-4b52-85d6-c2b53e6117b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:32.942233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1e725197-ee9f-4b52-85d6-c2b53e6117b1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:32.943446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1e725197-ee9f-4b52-85d6-c2b53e6117b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:33.034118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 67/143] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:33.201149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9798023-0ff5-44e6-b785-4be242728daf None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:33.201471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9798023-0ff5-44e6-b785-4be242728daf None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:33.201657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9798023-0ff5-44e6-b785-4be242728daf None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:33.203381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 77/144] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:33.208948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899245c-7f15-49de-afbe-c98c4ad4985f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:33.209277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899245c-7f15-49de-afbe-c98c4ad4985f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:33.209535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899245c-7f15-49de-afbe-c98c4ad4985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:33.232353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9899245c-7f15-49de-afbe-c98c4ad4985f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:33.233571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9899245c-7f15-49de-afbe-c98c4ad4985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:33.307770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 68/145] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:33.467308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9798023-0ff5-44e6-b785-4be242728daf None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:33.599795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9798023-0ff5-44e6-b785-4be242728daf None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:33.616033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:33.639203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:33.639838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:33.640117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:33.640388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:33.651345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:33.656345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:33.657181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:33.658376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 78/146] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:33.663612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899245c-7f15-49de-afbe-c98c4ad4985f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:33.682365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:33.690001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:33.799003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899245c-7f15-49de-afbe-c98c4ad4985f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:33.807253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29648e-e419-46ad-9739-8c1739f799d5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:33.838861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:33.839830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:33.840310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:33.840822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:33.841787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:33.842194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:33.843676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:33.844346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:33.845334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:33.845752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:33.846259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:33.853048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 69/147] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:33.862690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:33.867952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:33.869002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:33.869987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:33.872264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 79/148] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:33.874131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:33.877721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:33.947986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:34.006023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:34.051655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79fd74-6561-4cc6-9860-fb283875aecc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:34.094572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a146-86b3-424b-b9d0-008a90ef0746 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:34.098546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:34.099241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:34.099621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:34.099898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:34.101307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:34.103802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:34.105429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 70/149] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:33 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:34.111444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:34.162843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:34.163454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:34.163736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:34.164006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:34.164674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:34.164976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:34.166922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:34.184529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 80/150] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:34.189997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:34.255166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758c439-2c30-464e-9096-c1431d3ae89a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:34.298648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:34.299265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:34.303851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:34.304096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:34.336587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:34.362705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:34.374840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854944c-3b05-418a-9ed8-a9e6a724f623 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:34.423329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:34.423968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:34.424248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:34.424518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] PATH_INFO: `/v3/users/33023b1c1c8f45e1a9cea5b6f7c0f437` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:34.429634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:34.429932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] RBAC: Authorizing `identity:get_user(user_id=33023b1c1c8f45e1a9cea5b6f7c0f437)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:34.431971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:34.432365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:34.434063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 81/151] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:34 2022] GET /identity/v3/users/33023b1c1c8f45e1a9cea5b6f7c0f437 => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:34.440529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:34.551693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 71/152] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 14:43:34.558763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:34.569945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674f63-8c42-42e9-b04c-8d38e5afeac0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:34.609633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:34.610255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:34.610533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:34.610804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:34.612791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:34.614298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:34.615975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 82/153] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:34 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:34.623791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:34.649542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:34.686945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:34.782204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6c7f0-37c4-4c0a-842b-bef434742258 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:34.791941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58e1a9-46d0-4df9-8e38-cf1ed34af378 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:34.828531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:34.829117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:34.829420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:34.829695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/33023b1c1c8f45e1a9cea5b6f7c0f437/roles/5b6d374901c24586a24d2483221bed64` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:34.833484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] RBAC: Authorizing `identity:create_grant(project_id=4eabd47bff3c4b6eb0e462a0797efb23, user_id=33023b1c1c8f45e1a9cea5b6f7c0f437, role_id=5b6d374901c24586a24d2483221bed64)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:34.835172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:34.861789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 83/154] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:34 2022] PUT /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/users/33023b1c1c8f45e1a9cea5b6f7c0f437/roles/5b6d374901c24586a24d2483221bed64 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:34.872323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd42e5-4285-4d97-a348-fbac74bc79aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:34.873400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd42e5-4285-4d97-a348-fbac74bc79aa None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:34.873891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd42e5-4285-4d97-a348-fbac74bc79aa None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:34.874420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd42e5-4285-4d97-a348-fbac74bc79aa None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:34.876396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd42e5-4285-4d97-a348-fbac74bc79aa None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:34.878138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd42e5-4285-4d97-a348-fbac74bc79aa None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:34.880104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 72/155] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:34 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:34.887459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:35.017618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd5f3c-ab7c-4425-8c92-b6b4eab71870 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:35.020994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:35.079126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654ef1-dbab-4001-869e-288d07da08a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:35.079666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654ef1-dbab-4001-869e-288d07da08a5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:35.079943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654ef1-dbab-4001-869e-288d07da08a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:35.080212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654ef1-dbab-4001-869e-288d07da08a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:35.080950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654ef1-dbab-4001-869e-288d07da08a5 None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=55f51cbb4c144d569e357010b10f0ed6) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:35.081284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654ef1-dbab-4001-869e-288d07da08a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:35.082751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654ef1-dbab-4001-869e-288d07da08a5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:35.091634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 84/156] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:34 2022] GET /identity/v3/role_assignments?user.id=55f51cbb4c144d569e357010b10f0ed6&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:35.411080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649e518-8749-405d-9b9f-efaf7e5c181c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:35.411619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649e518-8749-405d-9b9f-efaf7e5c181c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:35.411619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649e518-8749-405d-9b9f-efaf7e5c181c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:35.413178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 73/157] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:35.419534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554159-42be-4f54-a2be-1dfff243add8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:35.419835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554159-42be-4f54-a2be-1dfff243add8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:35.420101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554159-42be-4f54-a2be-1dfff243add8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:35.443734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-eb554159-42be-4f54-a2be-1dfff243add8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:35.445328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-eb554159-42be-4f54-a2be-1dfff243add8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:35.464872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1a9bd-4bf5-49db-ad1d-773177790fcd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:35.465235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1a9bd-4bf5-49db-ad1d-773177790fcd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:35.465450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1a9bd-4bf5-49db-ad1d-773177790fcd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:35.467047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 74/158] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:35.473750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21aa1-8803-4d29-879d-3aedd87e2583 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:35.474088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21aa1-8803-4d29-879d-3aedd87e2583 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:35.474297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21aa1-8803-4d29-879d-3aedd87e2583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:35.493750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2cb21aa1-8803-4d29-879d-3aedd87e2583 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:35.495477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2cb21aa1-8803-4d29-879d-3aedd87e2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:35.563401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 85/159] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:35.595166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 75/160] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:35.843290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f273-480b-4f4d-8cbe-6a273d1cd2f3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:35.843475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f273-480b-4f4d-8cbe-6a273d1cd2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:35.843733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f273-480b-4f4d-8cbe-6a273d1cd2f3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:35.845076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 86/161] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:35.850792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d244d-cf6b-426b-ba94-a36237023d75 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:35.851079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d244d-cf6b-426b-ba94-a36237023d75 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:35.851352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d244d-cf6b-426b-ba94-a36237023d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:35.874283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-838d244d-cf6b-426b-ba94-a36237023d75 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:35.875482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-838d244d-cf6b-426b-ba94-a36237023d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:35.919520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:35.958524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 76/162] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:36.004981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3f273-480b-4f4d-8cbe-6a273d1cd2f3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:36.117738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:36.133888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3f273-480b-4f4d-8cbe-6a273d1cd2f3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:36.190104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:36.190740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:36.191017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:36.191281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:36.213891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:36.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:36.220196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:36.220990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 87/163] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:36.227942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d244d-cf6b-426b-ba94-a36237023d75 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:36.229030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:36.391080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb340436-bb4c-46b8-9475-3a3caca4af3f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:36.457953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:36.458606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:36.458856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:36.459166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:36.482067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:36.487009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:36.487902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:36.489034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 88/164] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:36.492053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d244d-cf6b-426b-ba94-a36237023d75 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:36.493857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:36.503493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:36.597899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:36.598517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:36.598796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:36.599068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:36.599731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:36.600028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:36.601433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:36.616501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 77/165] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:36.617474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:36.671707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16157a11-d42b-414e-bd8e-d88006af15d5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:36.707638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:36.723965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:36.724600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:36.724879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:36.725153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:36.736016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:36.740938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:36.741829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 89/166] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:36.743022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:36.844520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:36.867348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd73aa-b47e-414c-b1a6-41a5d2bd0294 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:36.882564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e72a9-5048-4327-9b38-1c8349865533 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:36.920950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:36.921957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:36.922176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:36.922450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] PATH_INFO: `/v3/users/3ec2e03066934f2597854e78a5c4d345` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:36.923655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] RBAC: Authorizing `identity:get_user(user_id=3ec2e03066934f2597854e78a5c4d345)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:36.925152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:36.926874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 90/167] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/users/3ec2e03066934f2597854e78a5c4d345 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:36.927640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:36.947666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:36.948354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:36.948634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:36.948902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:36.950887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:36.951191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:36.953313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:36.973078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 78/168] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:36.976122 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:37.168556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5021b1c-3834-490d-9fa2-dd9298617e80 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:37.218887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:37.235454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c334b-b426-4a77-8f47-5ee3efce5463 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:37.237112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:37.237631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:37.237907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:37.238173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:37.238857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:37.239152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:37.240575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:37.243416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:37.259160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 91/169] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:37.260280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:37.326833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb315e85-f9d9-48de-a16d-caf866bb0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:37.327461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb315e85-f9d9-48de-a16d-caf866bb0956 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:37.327744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb315e85-f9d9-48de-a16d-caf866bb0956 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:37.328019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb315e85-f9d9-48de-a16d-caf866bb0956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:37.439331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eab31a-b163-4a3e-aa71-c1bde6b13582 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:37.453187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb315e85-f9d9-48de-a16d-caf866bb0956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:37.454939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb315e85-f9d9-48de-a16d-caf866bb0956 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:37.514883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:37.515565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:37.515907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:37.516251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:37.517479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:37.518981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:37.520630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 92/170] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:37 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:37.526150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:37.563630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:37.581443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 79/171] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:37.582565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135473c-ce37-4aac-a21a-f7c70bdd4c3c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:37.582879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135473c-ce37-4aac-a21a-f7c70bdd4c3c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:37.585801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135473c-ce37-4aac-a21a-f7c70bdd4c3c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:37.589402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 80/172] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:37.590149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135473c-ce37-4aac-a21a-f7c70bdd4c3c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:37.660503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:37.737766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b7ac6-d995-4162-820a-4299182da4df None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:37.799160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135473c-ce37-4aac-a21a-f7c70bdd4c3c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:37.817070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:37.817783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:37.818128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:37.818404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] PATH_INFO: `/v3/users/55f51cbb4c144d569e357010b10f0ed6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:37.819616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] RBAC: Authorizing `identity:get_user(user_id=55f51cbb4c144d569e357010b10f0ed6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:37.821078 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:37.824342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 93/173] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:37 2022] GET /identity/v3/users/55f51cbb4c144d569e357010b10f0ed6 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:37.830089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:37.866448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:37.867061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:37.867787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:37.868063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:37.951096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b120a1-1043-44b2-b1f8-93f0913a05e6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:37.953670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:37.965632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:37.967085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:37.992914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:38.002516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0b919-165f-4a63-8b97-6ea9ccb57270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:38.003588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0b919-165f-4a63-8b97-6ea9ccb57270 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.004339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0b919-165f-4a63-8b97-6ea9ccb57270 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.005862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0b919-165f-4a63-8b97-6ea9ccb57270 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.007446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0b919-165f-4a63-8b97-6ea9ccb57270 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:38.018612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0b919-165f-4a63-8b97-6ea9ccb57270 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:38.021162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 94/174] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:37 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:38.023606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90522f-0010-4306-90ce-59510a024b86 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.025644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90522f-0010-4306-90ce-59510a024b86 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.025847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90522f-0010-4306-90ce-59510a024b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.049673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dc90522f-0010-4306-90ce-59510a024b86 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:38.052018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dc90522f-0010-4306-90ce-59510a024b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:38.092986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 81/175] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:38.094205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.140949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 95/176] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:38.140949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c1c6a-85bf-48af-b7ce-336e5d09467e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.140949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c1c6a-85bf-48af-b7ce-336e5d09467e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.140949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c1c6a-85bf-48af-b7ce-336e5d09467e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.140949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 96/177] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:38.141615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c1c6a-85bf-48af-b7ce-336e5d09467e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.203314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:38.209590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bae41-c535-4294-aeac-002a5755f246 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:38.256997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a5eea-a2e3-412c-b529-eeac13345396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:38.257644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5eea-a2e3-412c-b529-eeac13345396 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.258291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5eea-a2e3-412c-b529-eeac13345396 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.258577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5eea-a2e3-412c-b529-eeac13345396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.288977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c1c6a-85bf-48af-b7ce-336e5d09467e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:38.327455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:38.335580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:38.336196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.336476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.336762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.337538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=3ec2e03066934f2597854e78a5c4d345) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:38.337847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:38.339287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:38.343454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a5eea-a2e3-412c-b529-eeac13345396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:38.344888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a5eea-a2e3-412c-b529-eeac13345396 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:38.345265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 97/178] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/role_assignments?user.id=3ec2e03066934f2597854e78a5c4d345&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:38.346668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.459395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:38.466178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc976f-4adb-4a61-8a09-13d63f769b9f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:38.472484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 82/179] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:38.472484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95935429-a413-40f8-9ad4-b55fb0800cd4 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.472484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95935429-a413-40f8-9ad4-b55fb0800cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.472484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95935429-a413-40f8-9ad4-b55fb0800cd4 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.472484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 83/180] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:38.472484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f608-8062-4bfd-b229-a593e2ea9ed9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.473007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f608-8062-4bfd-b229-a593e2ea9ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.473007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f608-8062-4bfd-b229-a593e2ea9ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.494301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0941f608-8062-4bfd-b229-a593e2ea9ed9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:38.495557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0941f608-8062-4bfd-b229-a593e2ea9ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:38.512400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6bfb3-c63c-4c85-ab8b-c29a20879c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:38.513047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6bfb3-c63c-4c85-ab8b-c29a20879c1e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.513358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6bfb3-c63c-4c85-ab8b-c29a20879c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.513629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6bfb3-c63c-4c85-ab8b-c29a20879c1e None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.514774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6bfb3-c63c-4c85-ab8b-c29a20879c1e None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:38.516219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6bfb3-c63c-4c85-ab8b-c29a20879c1e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:38.522748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 98/181] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:38.522748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fae30-3992-4ee5-a89a-f55b6103b109 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.522748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fae30-3992-4ee5-a89a-f55b6103b109 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.522748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fae30-3992-4ee5-a89a-f55b6103b109 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.522748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 99/182] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:38.522748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fae30-3992-4ee5-a89a-f55b6103b109 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.584906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 84/183] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:38.585734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941f608-8062-4bfd-b229-a593e2ea9ed9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.601834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:38.644670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fae30-3992-4ee5-a89a-f55b6103b109 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:38.678267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458041e-a9cb-4508-a8d8-a55e1d458edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:38.678900 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458041e-a9cb-4508-a8d8-a55e1d458edf None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.679180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458041e-a9cb-4508-a8d8-a55e1d458edf None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.679449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458041e-a9cb-4508-a8d8-a55e1d458edf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.690286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458041e-a9cb-4508-a8d8-a55e1d458edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8458041e-a9cb-4508-a8d8-a55e1d458edf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:38.695165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:38.696041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:38.696823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 100/184] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:38.697505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0ba6-fdb0-4b7e-b31b-634f2d7d2ac7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.697774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0ba6-fdb0-4b7e-b31b-634f2d7d2ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.698053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0ba6-fdb0-4b7e-b31b-634f2d7d2ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.700226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941f608-8062-4bfd-b229-a593e2ea9ed9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:38.715255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-741e0ba6-fdb0-4b7e-b31b-634f2d7d2ac7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:38.716438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-741e0ba6-fdb0-4b7e-b31b-634f2d7d2ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:38.730470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:38.741773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:38.742563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.742847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.743117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] PATH_INFO: `/v3/projects/273dc016cd07438c81e7d37675d1daa4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.744773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] RBAC: Authorizing `identity:get_project(project_id=273dc016cd07438c81e7d37675d1daa4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:38.747188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:38.753716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 85/185] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/projects/273dc016cd07438c81e7d37675d1daa4 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:38.753716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.797232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 101/186] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:38.797232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48f8e-64c7-4407-8093-734a2fde4fe9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.797232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48f8e-64c7-4407-8093-734a2fde4fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.797232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48f8e-64c7-4407-8093-734a2fde4fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.813668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-08f48f8e-64c7-4407-8093-734a2fde4fe9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:38.815159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-08f48f8e-64c7-4407-8093-734a2fde4fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:38.872445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:38.875585 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e64c1a-0e41-448b-81ab-71d78056cc00 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:38.901748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 102/187] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:38.901748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f48f8e-64c7-4407-8093-734a2fde4fe9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.919079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:38.919819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:38.920096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:38.920365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:38.921138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] RBAC: Adding query filter params (role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=55f51cbb4c144d569e357010b10f0ed6) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:38.921510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:38.922947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:38.929874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 86/188] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/role_assignments?user.id=55f51cbb4c144d569e357010b10f0ed6&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:38.930427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:38.999155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.014791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f48f8e-64c7-4407-8093-734a2fde4fe9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.056978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3810-c76d-43ec-9209-e6a15d746f46 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.075744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.076527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.076740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.077019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.077728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:39.078027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:39.079442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:39.095937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.096867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.097462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.097935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.098841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=273dc016cd07438c81e7d37675d1daa4, user.id=55f51cbb4c144d569e357010b10f0ed6) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:39.099321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:39.099463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 103/189] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:39.104619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.106568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:39.113554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 87/190] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:38 2022] GET /identity/v3/role_assignments?user.id=55f51cbb4c144d569e357010b10f0ed6&scope.project.id=273dc016cd07438c81e7d37675d1daa4&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:39.115412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.198843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.274260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a6bf4-04c3-4594-bb26-290c7970acef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.285309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49257c-47c7-4e3c-9ef8-c83fca3c82d3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.314240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.331559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.332083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.332705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.332920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.333026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.333190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.333473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.334302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] PATH_INFO: `/v3/users/e283bc5896ae4225a973af12f9ace023` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.334760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:39.336473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:39.339557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 104/191] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:39.340421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.350822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] RBAC: Authorizing `identity:get_user(user_id=e283bc5896ae4225a973af12f9ace023)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:39.352362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:39.355809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 88/192] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/users/e283bc5896ae4225a973af12f9ace023 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:39.356380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.451639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fe05c-c7bf-4a8b-b158-3b69b549c5d8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.464752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.470466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae2922-a96d-4b65-98f7-aeda35ab2936 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.485092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.485963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.486198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.486468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.496771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:39.502045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:39.503109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 105/193] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:39.504003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.511351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.511996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.512271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.512534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.523837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:39.529299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:39.530173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:39.530987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 89/194] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:39.531632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.573245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.610101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387552f-858b-4398-97f5-2a7619857575 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.631750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e378f-c9a6-440d-82db-2971c3a1a086 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.642227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.642827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.643104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.643373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.644012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:39.645504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:39.666575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 106/195] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:43:39 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:39.666740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.667197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.667317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.667568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.667841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.723326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.747858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:39.749427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:39.796902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.804412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37745149-a88a-490c-8e26-1fbf4601a2bd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.840482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:39.841096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:39.841411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:39.841678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:39.842340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:39.842884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:39.844008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:39.853103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 107/196] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:39.858356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.866922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 90/197] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:39.867621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:39.983844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:39.994403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d7482-0c01-4d8a-bfac-94fd6dc2c915 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:39.996617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81239e-e39d-4d0c-a4ea-983ed71f2620 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:40.036995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.037649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.037916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.038186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.038854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:40.039150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.039688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.040322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.040570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.040739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.040977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] PATH_INFO: `/v3/users/e283bc5896ae4225a973af12f9ace023` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.042914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] RBAC: Authorizing `identity:get_user(user_id=e283bc5896ae4225a973af12f9ace023)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.044204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.046320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 91/198] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/users/e283bc5896ae4225a973af12f9ace023 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:40.047102 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:40.050671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 108/199] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:40.055089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:40.059369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:40.169639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf19d81-d793-49f8-aa52-d6176adec90a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:40.179193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886b06b-356b-4715-928a-78a5e5948529 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:40.209061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.209888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.210482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.210767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.212193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.213764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.215438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 92/200] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:40.216202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:40.223681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.224300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.224577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.224844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.253603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:40.316359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.318376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.320074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:40.346977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cb97a-38dd-4e7f-ada8-4701af1959f6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:40.385174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.386072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.386746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.387181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] PATH_INFO: `/v3/users/33023b1c1c8f45e1a9cea5b6f7c0f437` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.389249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] RBAC: Authorizing `identity:get_user(user_id=33023b1c1c8f45e1a9cea5b6f7c0f437)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.391112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.393265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 93/201] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/users/33023b1c1c8f45e1a9cea5b6f7c0f437 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:40.394403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:40.441045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 109/202] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:40.446595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:40.502753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:40.512712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930cda62-9f16-49de-b695-cbc36ff3490b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:40.555109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb612417-b48c-4bcc-b3b2-b6898c18c1ab None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:40.555339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.555723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.555998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.556790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.557557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=e283bc5896ae4225a973af12f9ace023) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:40.557859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.560575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.572485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 94/203] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/role_assignments?user.id=e283bc5896ae4225a973af12f9ace023&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:40.572485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:40.594218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:40.630371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.630997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.631272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.631540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.742114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9b8fe-8161-481f-8c8f-bf44a2c0743b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:40.781262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:40.781841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:40.782118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:40.782668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:40.783899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.785374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.787295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 95/204] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:40.794038 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:40.816870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:40.818422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:40.918488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:40.961545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:40.974245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 110/205] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:40.981120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:41.039231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65786d-049d-41b1-8c01-6f488adc2b6c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:41.098937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf13115-c399-491a-af0e-daf3ee65aa8f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:41.138348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:41.138589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda6c80-27b5-4ea9-8ce3-c43e17b0f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:41.139047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6c80-27b5-4ea9-8ce3-c43e17b0f50f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:41.139332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6c80-27b5-4ea9-8ce3-c43e17b0f50f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:41.139609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6c80-27b5-4ea9-8ce3-c43e17b0f50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:41.139907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:41.140383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:41.140499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6c80-27b5-4ea9-8ce3-c43e17b0f50f None admin] RBAC: Adding query filter params (role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=e283bc5896ae4225a973af12f9ace023) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:41.140888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:41.141005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6c80-27b5-4ea9-8ce3-c43e17b0f50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:41.142406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:41.142937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6c80-27b5-4ea9-8ce3-c43e17b0f50f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:41.144161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:41.146343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 111/206] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:41.152263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:41.163178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 96/207] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:40 2022] GET /identity/v3/role_assignments?user.id=e283bc5896ae4225a973af12f9ace023&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:41.197603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:41.283822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487b5c2-8860-4f0a-8f38-c13b05d63036 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:41.329588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b081b2-4a6b-415f-b775-ed01f9f32dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:41.330120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b081b2-4a6b-415f-b775-ed01f9f32dae None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:41.330459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b081b2-4a6b-415f-b775-ed01f9f32dae None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:41.330728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b081b2-4a6b-415f-b775-ed01f9f32dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:41.331524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b081b2-4a6b-415f-b775-ed01f9f32dae None admin] RBAC: Adding query filter params (role.id=5b6d374901c24586a24d2483221bed64, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23, user.id=33023b1c1c8f45e1a9cea5b6f7c0f437) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:41.331828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b081b2-4a6b-415f-b775-ed01f9f32dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:41.333308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b081b2-4a6b-415f-b775-ed01f9f32dae None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:41.340776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 112/208] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:41 2022] GET /identity/v3/role_assignments?user.id=33023b1c1c8f45e1a9cea5b6f7c0f437&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=5b6d374901c24586a24d2483221bed64 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:42.145121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7172cc-47b6-44a3-9ca0-8d5b6e7fc0ab None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.145669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7172cc-47b6-44a3-9ca0-8d5b6e7fc0ab None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.145669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7172cc-47b6-44a3-9ca0-8d5b6e7fc0ab None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.147148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 97/209] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:42.153952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97b554-264e-42e7-aeac-e426e88325fd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.154718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97b554-264e-42e7-aeac-e426e88325fd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.154718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97b554-264e-42e7-aeac-e426e88325fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.172920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ec97b554-264e-42e7-aeac-e426e88325fd None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:42.174135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ec97b554-264e-42e7-aeac-e426e88325fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:42.243103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 113/210] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:42.256774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386af080-2038-4acb-a622-68f004dad600 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.257352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386af080-2038-4acb-a622-68f004dad600 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.257810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386af080-2038-4acb-a622-68f004dad600 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.259965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 98/211] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:42.265766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0443c-e323-43d7-bd15-8084dfc5d609 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.266247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0443c-e323-43d7-bd15-8084dfc5d609 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.266703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0443c-e323-43d7-bd15-8084dfc5d609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.285467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-50c0443c-e323-43d7-bd15-8084dfc5d609 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:42.286937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-50c0443c-e323-43d7-bd15-8084dfc5d609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:42.298019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:42.358797 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 114/212] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:42.519723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386af080-2038-4acb-a622-68f004dad600 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:42.624863 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386af080-2038-4acb-a622-68f004dad600 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:42.641443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0443c-e323-43d7-bd15-8084dfc5d609 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:42.665140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:42.666496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.666776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.667048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.677181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:42.686052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:42.691086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:42.691954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:42.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 99/213] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:42.702305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:42.759057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0443c-e323-43d7-bd15-8084dfc5d609 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:42.801508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:42.802266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.802615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.802982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.804332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:42.806112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:42.812480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 115/214] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:42.813773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:42.830046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:42.832762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37e432-e352-4831-ad10-5bd49a467241 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:42.882032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b42ed6-8ff4-4dcc-b925-70b41628b6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:42.882633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b42ed6-8ff4-4dcc-b925-70b41628b6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.883023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b42ed6-8ff4-4dcc-b925-70b41628b6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.883440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b42ed6-8ff4-4dcc-b925-70b41628b6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.884324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b42ed6-8ff4-4dcc-b925-70b41628b6a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:42.884717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b42ed6-8ff4-4dcc-b925-70b41628b6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:42.886636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b42ed6-8ff4-4dcc-b925-70b41628b6a9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:42.895758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 100/215] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:42.897026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c815-1356-4563-a9fa-6c4a2cb7cff7 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.897412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c815-1356-4563-a9fa-6c4a2cb7cff7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.897746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c815-1356-4563-a9fa-6c4a2cb7cff7 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.899168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 101/216] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:42.903999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396c815-1356-4563-a9fa-6c4a2cb7cff7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:42.927936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce33876-8905-4763-b525-5677d6bc2239 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:42.961087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c1b1d-20d4-493b-8c79-b0a402f605ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:42.961720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c1b1d-20d4-493b-8c79-b0a402f605ee None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.962002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c1b1d-20d4-493b-8c79-b0a402f605ee None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.962783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c1b1d-20d4-493b-8c79-b0a402f605ee None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:42.963437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c1b1d-20d4-493b-8c79-b0a402f605ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:42.964894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c1b1d-20d4-493b-8c79-b0a402f605ee None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:42.975545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:42.991030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 116/217] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:43:42 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:43:42.998858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc190ae2-a920-4385-a19d-cec76da43311 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:42.999157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc190ae2-a920-4385-a19d-cec76da43311 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:42.999430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc190ae2-a920-4385-a19d-cec76da43311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.021473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fc190ae2-a920-4385-a19d-cec76da43311 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:43.022707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fc190ae2-a920-4385-a19d-cec76da43311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:43.044947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396c815-1356-4563-a9fa-6c4a2cb7cff7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:43.092364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:43.093782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.094814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.095555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] PATH_INFO: `/v3/users/a775cb39facd44ca958b24d069b49eae` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.096636 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:43.108312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 117/218] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:43.119611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] RBAC: Authorizing `identity:get_user(user_id=a775cb39facd44ca958b24d069b49eae)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:43.121119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:43.122916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 102/219] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:42 2022] GET /identity/v3/users/a775cb39facd44ca958b24d069b49eae => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:43.128699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc190ae2-a920-4385-a19d-cec76da43311 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:43.241307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc190ae2-a920-4385-a19d-cec76da43311 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:43.279179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:43.279800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.280080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.280351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.353765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:43.358770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:43.360279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:43.388386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:43.479358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 118/220] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:43.487002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:43.540141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c2fd9-3e6d-4cad-aced-a20833d01047 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:43.590986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:43.609113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:43.613949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa53c07-dced-434b-b654-ce5f00cbf6e7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:43.650526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:43.651186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.651462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.651730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.655483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:43.656173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.656459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.656736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.657895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:43.659666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:43.665302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 119/221] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:43 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:43.668715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:43.678763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:43.683992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:43.684833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:43.685664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:43.693107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 103/222] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:43.694031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:43.834041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4653-29ba-47a4-903c-1450eb1da10a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:43.837820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73ca0-4d9a-418e-bde4-fb2d354d89f3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:43.870487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c629d5a-8a25-49f7-a6f4-3bf5b011d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:43.871382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c629d5a-8a25-49f7-a6f4-3bf5b011d3fb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.871662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c629d5a-8a25-49f7-a6f4-3bf5b011d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.871933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c629d5a-8a25-49f7-a6f4-3bf5b011d3fb None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.872599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c629d5a-8a25-49f7-a6f4-3bf5b011d3fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:43.872901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c629d5a-8a25-49f7-a6f4-3bf5b011d3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:43.874337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c629d5a-8a25-49f7-a6f4-3bf5b011d3fb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:43.874999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9d947-4077-4506-9930-e7b5b9968316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:43.875616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d947-4077-4506-9930-e7b5b9968316 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.875898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d947-4077-4506-9930-e7b5b9968316 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.876172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d947-4077-4506-9930-e7b5b9968316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.876894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9d947-4077-4506-9930-e7b5b9968316 None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=a775cb39facd44ca958b24d069b49eae) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:43.877216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9d947-4077-4506-9930-e7b5b9968316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:43.878733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9d947-4077-4506-9930-e7b5b9968316 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:43.883637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 104/223] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:43.885157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 120/224] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:43 2022] GET /identity/v3/role_assignments?user.id=a775cb39facd44ca958b24d069b49eae&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:43.885362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1e8e2-9945-4cb5-b5dc-94301b3360bd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.885768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1e8e2-9945-4cb5-b5dc-94301b3360bd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.886042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1e8e2-9945-4cb5-b5dc-94301b3360bd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.887652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 105/225] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:43.894779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7b05-adad-4657-9315-5e5934766de4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:43.895059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7b05-adad-4657-9315-5e5934766de4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:43.895333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7b05-adad-4657-9315-5e5934766de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:43.899503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1e8e2-9945-4cb5-b5dc-94301b3360bd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:43.940474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cc1d7b05-adad-4657-9315-5e5934766de4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:43.941737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cc1d7b05-adad-4657-9315-5e5934766de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:43.999640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:44.013597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:44.050361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 121/226] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:44.151117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1e8e2-9945-4cb5-b5dc-94301b3360bd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:44.205911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:44.206960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:44.207445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:44.207955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] PATH_INFO: `/v3/users/e283bc5896ae4225a973af12f9ace023` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:44.209372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=e283bc5896ae4225a973af12f9ace023)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:44.211010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:44.213505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 106/227] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:43 2022] GET /identity/v3/users/e283bc5896ae4225a973af12f9ace023 => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:44.219368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d7b05-adad-4657-9315-5e5934766de4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:44.362065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d7b05-adad-4657-9315-5e5934766de4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:44.375856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:44.407301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:44.407862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:44.408139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:44.408407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:44.409699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:44.411303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:44.412936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 122/228] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:44 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:44.419017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:44.457552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:44.533884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f43d7a-13cd-4b4b-bd88-4e8a5c96f62f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:44.571003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:44.571759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:44.572553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:44.572827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/e283bc5896ae4225a973af12f9ace023/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:44.575324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] RBAC: Authorizing `identity:create_grant(project_id=9b97fbead9af485db8541ad29cc7fe55, user_id=e283bc5896ae4225a973af12f9ace023, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:44.577156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:44.592909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 123/229] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:44 2022] PUT /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/e283bc5896ae4225a973af12f9ace023/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:43:44.685415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b075a26-5988-4e4e-9e3b-1b0c28096bd0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:44.707015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:44.739969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:44.740636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:44.740916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:44.741190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:44.760154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:44.766018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:44.766910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:44.767705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 107/230] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:43:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:43:44.773098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:44.876158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:44.889185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60110a91-1b9a-4b90-bdf0-ca87e940807f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:44.924819 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:44.925503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:44.925790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:44.926065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:44.926729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:44.927083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:44.928465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:44.937383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 124/231] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:43:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:44.943049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:45.091317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd935-8925-4614-9ca2-4c0c532ce35c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:45.145493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:45.146473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:45.146942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:45.147423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] PATH_INFO: `/v3/users/e283bc5896ae4225a973af12f9ace023` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:45.153861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] RBAC: Authorizing `identity:get_user(user_id=e283bc5896ae4225a973af12f9ace023)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:45.155614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:45.157504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 108/232] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:44 2022] GET /identity/v3/users/e283bc5896ae4225a973af12f9ace023 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:45.163333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:45.329987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a88a25-c2f0-4e7a-9140-a5184b6bd1ab None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:45.374943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066c685-ac9a-416b-a4b0-9d343f674b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:45.376347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066c685-ac9a-416b-a4b0-9d343f674b29 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:45.376827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066c685-ac9a-416b-a4b0-9d343f674b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:45.377313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066c685-ac9a-416b-a4b0-9d343f674b29 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:45.381947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066c685-ac9a-416b-a4b0-9d343f674b29 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:45.383682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066c685-ac9a-416b-a4b0-9d343f674b29 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:45.386649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 125/233] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:45 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:45.392025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:45.439293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:45.514109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577bfb-0d47-45a5-b3d6-c3b71820f019 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:45.570044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff7d5c-7ddb-4fbc-a9ac-664873cee85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:45.571128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff7d5c-7ddb-4fbc-a9ac-664873cee85c None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:45.572145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff7d5c-7ddb-4fbc-a9ac-664873cee85c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:45.572639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff7d5c-7ddb-4fbc-a9ac-664873cee85c None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/e283bc5896ae4225a973af12f9ace023/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:45.575449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff7d5c-7ddb-4fbc-a9ac-664873cee85c None admin] RBAC: Authorizing `identity:create_grant(project_id=9b97fbead9af485db8541ad29cc7fe55, user_id=e283bc5896ae4225a973af12f9ace023, role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:45.577260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff7d5c-7ddb-4fbc-a9ac-664873cee85c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:45.595621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 109/234] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:45 2022] PUT /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/e283bc5896ae4225a973af12f9ace023/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:45.721376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7bc73-dfc3-433b-bfe1-31cb029bbcda None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:45.722009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7bc73-dfc3-433b-bfe1-31cb029bbcda None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:45.722500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7bc73-dfc3-433b-bfe1-31cb029bbcda None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:45.724036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 126/235] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:45.729643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccae7-0223-471e-8c0b-28ee7fa2697c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:45.730568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccae7-0223-471e-8c0b-28ee7fa2697c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:45.731162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccae7-0223-471e-8c0b-28ee7fa2697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:45.754338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6f9ccae7-0223-471e-8c0b-28ee7fa2697c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:45.755964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6f9ccae7-0223-471e-8c0b-28ee7fa2697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:45.845051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 110/236] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:45.919109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34114ef7-d6fb-4081-9879-9a92c0043bb8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:45.919628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34114ef7-d6fb-4081-9879-9a92c0043bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:45.920043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34114ef7-d6fb-4081-9879-9a92c0043bb8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:45.921565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 127/237] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:45.926833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b0ef0-5b93-4cff-ac9f-bbc9036b1eaa None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:45.927115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b0ef0-5b93-4cff-ac9f-bbc9036b1eaa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:45.927387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b0ef0-5b93-4cff-ac9f-bbc9036b1eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:45.949787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-021b0ef0-5b93-4cff-ac9f-bbc9036b1eaa None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:45.951013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-021b0ef0-5b93-4cff-ac9f-bbc9036b1eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:45.980192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:46.032951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 111/238] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:46.166716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34114ef7-d6fb-4081-9879-9a92c0043bb8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:46.190074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:46.305755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34114ef7-d6fb-4081-9879-9a92c0043bb8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:46.342422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:46.343063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:46.343338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:46.343606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:46.354542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:46.359524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:46.360370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:46.361419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 128/239] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:46.368023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b0ef0-5b93-4cff-ac9f-bbc9036b1eaa None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:46.473287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:46.499363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b0ef0-5b93-4cff-ac9f-bbc9036b1eaa None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:46.570588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:46.571198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:46.571470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:46.572293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:46.572957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:46.573279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:46.574637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:46.584413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 112/240] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:46.590748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:46.633133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:46.649104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:46.701687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec038e-d14e-4b77-881b-f67b3c7b42db None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:46.755290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7feb951-2992-46b8-a9eb-70460f28a5b2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:46.777107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:46.779245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:46.782247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:46.782711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:46.785563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:46.788103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:46.792154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 129/241] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:46.802516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:46.844272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa453243-12b5-4aac-841b-77de2905c9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:46.844948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa453243-12b5-4aac-841b-77de2905c9de None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:46.845216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa453243-12b5-4aac-841b-77de2905c9de None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:46.845499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa453243-12b5-4aac-841b-77de2905c9de None admin] PATH_INFO: `/v3/users/bcabe562523e46ed972fdc4c06f45731` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:46.864598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa453243-12b5-4aac-841b-77de2905c9de None admin] RBAC: Authorizing `identity:get_user(user_id=bcabe562523e46ed972fdc4c06f45731)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:46.866104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa453243-12b5-4aac-841b-77de2905c9de None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:46.868597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 113/242] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:46 2022] GET /identity/v3/users/bcabe562523e46ed972fdc4c06f45731 => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:46.870449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4851e46-934a-465c-bdcb-e5225fb2f06e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:46.870733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4851e46-934a-465c-bdcb-e5225fb2f06e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:46.871007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4851e46-934a-465c-bdcb-e5225fb2f06e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:46.876965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 114/243] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:46.876965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4851e46-934a-465c-bdcb-e5225fb2f06e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:46.990951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82bd2d-fc52-4436-86e3-6f46df440cd8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:47.032935 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8150b7-e2ea-49da-9c37-be8aa5c3f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:47.033576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8150b7-e2ea-49da-9c37-be8aa5c3f9e5 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.033857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8150b7-e2ea-49da-9c37-be8aa5c3f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.034130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8150b7-e2ea-49da-9c37-be8aa5c3f9e5 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.034782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8150b7-e2ea-49da-9c37-be8aa5c3f9e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:47.036237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8150b7-e2ea-49da-9c37-be8aa5c3f9e5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:47.042379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:47.060044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 130/244] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:43:46 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:47.061478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d61da-8c8a-4112-ab04-69c0beecaa71 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.062120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d61da-8c8a-4112-ab04-69c0beecaa71 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.062398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d61da-8c8a-4112-ab04-69c0beecaa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.069094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:47.087019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5d4d61da-8c8a-4112-ab04-69c0beecaa71 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:47.089020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5d4d61da-8c8a-4112-ab04-69c0beecaa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:47.103354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4851e46-934a-465c-bdcb-e5225fb2f06e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:47.175333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 131/245] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:47.221735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:47.223746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.224048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.224321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.242944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9242d4f-fa65-4bea-8307-af7ce0d6c6ff None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.244207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9242d4f-fa65-4bea-8307-af7ce0d6c6ff None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.244750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9242d4f-fa65-4bea-8307-af7ce0d6c6ff None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.246752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 132/246] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:47.258301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fcc2-7bcd-4f67-b30c-4028571f6731 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.258932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fcc2-7bcd-4f67-b30c-4028571f6731 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.259944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fcc2-7bcd-4f67-b30c-4028571f6731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.294604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0018fcc2-7bcd-4f67-b30c-4028571f6731 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:47.296862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0018fcc2-7bcd-4f67-b30c-4028571f6731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:47.389951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:47.391861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:47.443270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 133/247] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:47.514424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fcc2-7bcd-4f67-b30c-4028571f6731 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:47.528975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:47.568954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 115/248] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:47.576613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:47.586558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:47.641561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fcc2-7bcd-4f67-b30c-4028571f6731 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:47.692702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:47.693709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.694209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.694698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.707935 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:47.711291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46d152-14d0-4fe0-9f11-4a7f885fe005 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:47.713603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:47.714468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:47.715231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:47.717533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 134/249] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:43:47.724356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:47.752891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:47.753757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.754044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.754328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.755618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:47.757758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:47.759989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 116/250] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:47 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:47.761445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:47.820516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:47.859763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9518fce-65a6-433a-8384-b205642b9a7f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:47.889576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfcd41-f17f-4237-91b5-1aa8cf69d6fa None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:47.894046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:47.905529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:47.906143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.906582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.906855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.907906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:47.908341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:47.910038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:47.921096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 135/251] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:47.921958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:47.934905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:47.935595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:47.935871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:47.936233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:47.948270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:47.954336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:47.955187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:43:47.956011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:47.956610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 117/252] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:47.958248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:48.080377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e8bb2-2c1a-461e-aa5b-4669fd4fb4db None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:48.128668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56ee15-0497-4a08-87a8-a1a4a6e29972 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:48.141568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:48.153109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:48.153864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.154148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.154419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.155152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] RBAC: Adding query filter params (role.id=5b6d374901c24586a24d2483221bed64, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=bcabe562523e46ed972fdc4c06f45731) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:48.155456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:48.156911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:48.167566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:48.168201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.168477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.168754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] PATH_INFO: `/v3/users/a775cb39facd44ca958b24d069b49eae` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.169925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] RBAC: Authorizing `identity:get_user(user_id=a775cb39facd44ca958b24d069b49eae)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:48.171380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:48.173273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 118/253] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:47 2022] GET /identity/v3/users/a775cb39facd44ca958b24d069b49eae => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:48.174347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:48.179212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 136/254] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:47 2022] GET /identity/v3/role_assignments?user.id=bcabe562523e46ed972fdc4c06f45731&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=5b6d374901c24586a24d2483221bed64 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:48.180077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:48.284599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88bc96-4cae-4067-b0a0-233a5785caad None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:48.318453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:48.324452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:48.325107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.325440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.325712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.326415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:48.326715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:48.328149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:48.338732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 119/255] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:48.347193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:48.393095 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:48.460582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94b92-2c05-4296-b716-b6921f050d4f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:48.483176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc21c-3691-46ba-ad82-bca3cc82e36e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:48.519312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b0e04-97f0-4647-9412-e308efc443d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:48.520021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b0e04-97f0-4647-9412-e308efc443d8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.520300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b0e04-97f0-4647-9412-e308efc443d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.520572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b0e04-97f0-4647-9412-e308efc443d8 None admin] PATH_INFO: `/v3/users/e283bc5896ae4225a973af12f9ace023` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.521804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b0e04-97f0-4647-9412-e308efc443d8 None admin] RBAC: Authorizing `identity:get_user(user_id=e283bc5896ae4225a973af12f9ace023)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:48.523807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b0e04-97f0-4647-9412-e308efc443d8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:48.525809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 120/256] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:48 2022] GET /identity/v3/users/e283bc5896ae4225a973af12f9ace023 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:48.530032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dad50d-9521-4727-8e34-fcf4fb1d92f1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.530312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dad50d-9521-4727-8e34-fcf4fb1d92f1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.530666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dad50d-9521-4727-8e34-fcf4fb1d92f1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.532073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 121/257] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:48.532653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dad50d-9521-4727-8e34-fcf4fb1d92f1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:48.563033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46828e-f787-4b9b-9efe-33c76ca2e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:48.563768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46828e-f787-4b9b-9efe-33c76ca2e147 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.564050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46828e-f787-4b9b-9efe-33c76ca2e147 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.564326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46828e-f787-4b9b-9efe-33c76ca2e147 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.569928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46828e-f787-4b9b-9efe-33c76ca2e147 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:48.571708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46828e-f787-4b9b-9efe-33c76ca2e147 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:48.573561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 137/258] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:48 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:48.574525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d60a-e49a-4092-bd50-ac04ea676dd6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.574804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d60a-e49a-4092-bd50-ac04ea676dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.575192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d60a-e49a-4092-bd50-ac04ea676dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.612042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1381d60a-e49a-4092-bd50-ac04ea676dd6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:48.614360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1381d60a-e49a-4092-bd50-ac04ea676dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:48.701319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 138/259] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:48.702026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381d60a-e49a-4092-bd50-ac04ea676dd6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:48.717871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dad50d-9521-4727-8e34-fcf4fb1d92f1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:48.723152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:48.778736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:48.801961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:48.802632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.802861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.803128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:48.901437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381d60a-e49a-4092-bd50-ac04ea676dd6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:48.941698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:48.943184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:48.997927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:48.998609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:48.998887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:48.999160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/a775cb39facd44ca958b24d069b49eae/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:49.001941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b97fbead9af485db8541ad29cc7fe55, user_id=a775cb39facd44ca958b24d069b49eae, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:49.003399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:49.033377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 139/260] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:48 2022] PUT /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/a775cb39facd44ca958b24d069b49eae/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:49.094369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:49.130173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 122/261] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:49.136392 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:49.142268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:49.294404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf0ea3-8eef-49d6-8abe-83b8f6f5fdf1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:49.305089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:49.370604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:49.371241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:49.371520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:49.371789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:49.390352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8da67-72db-483b-8993-dc34768cd539 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:49.392367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:49.397430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:49.398288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:49.399063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 140/262] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:43:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:43:49.418656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:49.431597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:49.432232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:49.432504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:49.432769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:49.435782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:49.437262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:49.439268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 123/263] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:49 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:49.445326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:49.533422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:49.624892 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08252352-962e-42ec-93f7-7fd05219e3c3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:49.633672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46486ac-db91-427f-851d-45e7c841c692 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:49.698812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:49.699560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:49.699840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:49.700105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:49.700766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:49.701061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:49.702835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:49.712025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 141/264] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:43:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:49.719836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:49.729901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:49.730952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:49.731238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:49.731513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:49.732249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=e283bc5896ae4225a973af12f9ace023) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:49.732556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:49.734027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:49.747616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 124/265] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:49 2022] GET /identity/v3/role_assignments?user.id=e283bc5896ae4225a973af12f9ace023&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:49.766765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:49.876142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9d819-776f-47fb-91ed-4cd60fab80a3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:49.929381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e8be0-4c0e-4631-9aab-ced3961b000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:49.930033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8be0-4c0e-4631-9aab-ced3961b000d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:49.930311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8be0-4c0e-4631-9aab-ced3961b000d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:49.930584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8be0-4c0e-4631-9aab-ced3961b000d None admin] PATH_INFO: `/v3/users/e283bc5896ae4225a973af12f9ace023` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:49.935218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e8be0-4c0e-4631-9aab-ced3961b000d None admin] RBAC: Authorizing `identity:get_user(user_id=e283bc5896ae4225a973af12f9ace023)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:49.936730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e8be0-4c0e-4631-9aab-ced3961b000d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:49.938787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 142/266] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:49 2022] GET /identity/v3/users/e283bc5896ae4225a973af12f9ace023 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:49.944192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:50.099847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7043a5-722f-4af9-b1d2-a3fabd82d316 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:50.151150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:50.151684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:50.151961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:50.152231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:50.155848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:50.235117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:50.236675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:50.305246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf22286-8875-4097-aef6-279a749aadc1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:50.305615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf22286-8875-4097-aef6-279a749aadc1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:50.305999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf22286-8875-4097-aef6-279a749aadc1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:50.307646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 143/267] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:50.312621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60703-c6d8-46c2-a4d5-1224a2246240 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:50.312973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60703-c6d8-46c2-a4d5-1224a2246240 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:50.313357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60703-c6d8-46c2-a4d5-1224a2246240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:50.334370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7fd60703-c6d8-46c2-a4d5-1224a2246240 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:50.335638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7fd60703-c6d8-46c2-a4d5-1224a2246240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:50.341484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 125/268] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:50.350568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:50.397707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:50.458574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c75f8-d8ea-4a76-b979-1cf9e2ee2eb2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:50.486749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 144/269] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:50.501441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:50.502088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:50.502361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:50.502629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:50.508497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:50.510071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:50.512461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 126/270] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:50 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:50.518442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd60703-c6d8-46c2-a4d5-1224a2246240 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:50.550114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:50.633516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd60703-c6d8-46c2-a4d5-1224a2246240 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:50.670229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca53cf2-0143-4362-8bc6-37c12d02bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:50.670860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53cf2-0143-4362-8bc6-37c12d02bfcc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:50.671135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53cf2-0143-4362-8bc6-37c12d02bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:50.671379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53cf2-0143-4362-8bc6-37c12d02bfcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:50.672116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca53cf2-0143-4362-8bc6-37c12d02bfcc None admin] RBAC: Adding query filter params (role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=e283bc5896ae4225a973af12f9ace023) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:50.672420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca53cf2-0143-4362-8bc6-37c12d02bfcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:50.673934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca53cf2-0143-4362-8bc6-37c12d02bfcc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:50.681077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 145/271] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:50 2022] GET /identity/v3/role_assignments?user.id=e283bc5896ae4225a973af12f9ace023&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:50.847891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:50.875624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1ae8c-edee-4a69-8ca2-8ed2dc60eac1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:50.875946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1ae8c-edee-4a69-8ca2-8ed2dc60eac1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:50.876183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1ae8c-edee-4a69-8ca2-8ed2dc60eac1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:50.877912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 146/272] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:50.882878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f048-33ce-46fb-95ba-f0fc82dc30a3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:50.883261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f048-33ce-46fb-95ba-f0fc82dc30a3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:50.883538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f048-33ce-46fb-95ba-f0fc82dc30a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:50.918372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ef51f048-33ce-46fb-95ba-f0fc82dc30a3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:50.919603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ef51f048-33ce-46fb-95ba-f0fc82dc30a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:50.960188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70007f-64ac-4893-9610-846e9c4a8379 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:50.963227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:50.999780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.000417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.000693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.000958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.012394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:51.021289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:51.022188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:51.022960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 127/273] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:51.030288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:51.056964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 147/274] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:51.146060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ed956-80b8-4236-9ead-c96ec9c4352b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:51.176253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12df6-a610-40c0-963e-14db10381f1a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.176520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12df6-a610-40c0-963e-14db10381f1a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.176792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12df6-a610-40c0-963e-14db10381f1a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.181978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 148/275] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:51.183898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b18d0-8cb3-4830-b7ba-5404bbd251b3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.184178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b18d0-8cb3-4830-b7ba-5404bbd251b3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.184450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b18d0-8cb3-4830-b7ba-5404bbd251b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.194017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.194934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.195401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.195863 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.197049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:51.197587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:51.199323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:51.206667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6a1b18d0-8cb3-4830-b7ba-5404bbd251b3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:51.208171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a1b18d0-8cb3-4830-b7ba-5404bbd251b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:51.210127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 128/276] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:51.216409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:51.229874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:51.257580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:51.308993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 149/277] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:51.346682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b18d0-8cb3-4830-b7ba-5404bbd251b3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:51.349510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbd5db-6474-4fcb-b811-f2108451d284 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:51.391323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.391958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.392234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.392504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] PATH_INFO: `/v3/users/dae4351a1a85452c99a4e30821f80cd8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.408560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] RBAC: Authorizing `identity:get_user(user_id=dae4351a1a85452c99a4e30821f80cd8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:51.410137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:51.412086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 129/278] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:51 2022] GET /identity/v3/users/dae4351a1a85452c99a4e30821f80cd8 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:51.417776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:51.470107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b18d0-8cb3-4830-b7ba-5404bbd251b3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:51.514114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:51.516105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.516747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.517144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.517471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.518607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:51.520056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:51.526088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 150/279] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:51.528543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:51.539110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e7e34-60e9-4470-9311-69867211ce41 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:51.553480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:51.581684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.582364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.582645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.582916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.583694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, user.id=dae4351a1a85452c99a4e30821f80cd8, scope.system=all) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:51.584003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:51.585459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:51.593294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 130/280] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 14:43:51 2022] GET /identity/v3/role_assignments?user.id=dae4351a1a85452c99a4e30821f80cd8&scope.system=all&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:51.622867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:51.673661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834df69-6c5e-477f-8011-13a9224f3a4c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:51.711425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.712075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.712353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.712617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.713372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:51.714857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:51.743082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 151/281] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:43:51 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:43:51.762246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c81be-1c64-47ca-9873-dd4216e9370c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:51.805248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.806413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.806893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.807370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.819988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:51.825443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:51.826449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:51.827248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:51.830067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 131/282] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:43:51.839105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:51.927353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:51.955489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67e56e-9651-4782-ab25-9940c623a02c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:51.991500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591df26-ce66-40aa-a00a-075c7b354e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:51.992135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591df26-ce66-40aa-a00a-075c7b354e50 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:51.992434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591df26-ce66-40aa-a00a-075c7b354e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:51.992703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591df26-ce66-40aa-a00a-075c7b354e50 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:51.995826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591df26-ce66-40aa-a00a-075c7b354e50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:51.995826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591df26-ce66-40aa-a00a-075c7b354e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:51.995826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591df26-ce66-40aa-a00a-075c7b354e50 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:52.013131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 152/283] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:52.015900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:52.048864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:52.120322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf5c3e-af2a-4c93-9cc4-885021284b23 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:52.120659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf5c3e-af2a-4c93-9cc4-885021284b23 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:52.120874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf5c3e-af2a-4c93-9cc4-885021284b23 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:52.122258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 153/284] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:52.128643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c21c8c-572e-4cbb-b057-6d2fa0900d1c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:52.129811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b98d8-d0b8-4a46-b0ee-d41c5e233518 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:52.130476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b98d8-d0b8-4a46-b0ee-d41c5e233518 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:52.131805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b98d8-d0b8-4a46-b0ee-d41c5e233518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:52.156637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3f8b98d8-d0b8-4a46-b0ee-d41c5e233518 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:52.157861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3f8b98d8-d0b8-4a46-b0ee-d41c5e233518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:52.168400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:52.169040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:52.169350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:52.169623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] PATH_INFO: `/v3/users/bcabe562523e46ed972fdc4c06f45731` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:52.171120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] RBAC: Authorizing `identity:get_user(user_id=bcabe562523e46ed972fdc4c06f45731)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:52.172885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:52.174769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 132/285] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:52 2022] GET /identity/v3/users/bcabe562523e46ed972fdc4c06f45731 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:52.180220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:52.241325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 154/286] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:52.269301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:52.295937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88f0a3-eb4e-4b85-8904-dc2f1ae448f7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:52.332883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:52.333969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:52.334532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:52.334999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:52.336327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:52.338115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:52.339956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 133/287] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:52 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:52.345712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b98d8-d0b8-4a46-b0ee-d41c5e233518 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:52.468920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:52.520484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b98d8-d0b8-4a46-b0ee-d41c5e233518 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:52.529398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:52.567602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:52.568276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:52.568555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:52.568831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/bcabe562523e46ed972fdc4c06f45731/roles/5b6d374901c24586a24d2483221bed64` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:52.572241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b97fbead9af485db8541ad29cc7fe55, user_id=bcabe562523e46ed972fdc4c06f45731, role_id=5b6d374901c24586a24d2483221bed64)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:52.573717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:52.595484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 155/288] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:43:52 2022] PUT /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55/users/bcabe562523e46ed972fdc4c06f45731/roles/5b6d374901c24586a24d2483221bed64 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:52.694877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b4cca-b24e-48e1-8394-3d36490322c9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:52.747867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:52.748444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:52.748724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:52.748995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:52.768673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:52.773991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:52.775539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:52.776383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 134/289] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:52.783625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:52.923238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011f1fc-d269-4b4e-a43a-0b4a0f9b26a8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:52.950870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:52.961019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:52.962502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:52.962804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:52.963079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:52.963760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:52.964064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:52.965495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:52.976141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 156/290] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:52.982412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:52.989330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:53.130965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddecd1-4a6f-454b-97b7-611c2dffabf1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:53.171606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b981122-5822-4388-a696-45033368f51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:53.172508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b981122-5822-4388-a696-45033368f51d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:53.172963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b981122-5822-4388-a696-45033368f51d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:53.173437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b981122-5822-4388-a696-45033368f51d None admin] PATH_INFO: `/v3/users/a775cb39facd44ca958b24d069b49eae` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:53.174769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b981122-5822-4388-a696-45033368f51d None admin] RBAC: Authorizing `identity:get_user(user_id=a775cb39facd44ca958b24d069b49eae)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:53.176543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b981122-5822-4388-a696-45033368f51d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:53.178631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 135/291] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:52 2022] GET /identity/v3/users/a775cb39facd44ca958b24d069b49eae => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:53.184441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:53.315216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b84a19-1934-40e7-9df7-c18a57e08d97 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:53.366018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:53.366630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:53.366912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:53.367185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:53.447947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:53.465154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:53.466968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:53.594202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 157/292] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:53.600257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b981122-5822-4388-a696-45033368f51d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:53.706181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:53.748007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b981122-5822-4388-a696-45033368f51d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:53.793622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26279163-7666-4fa3-8c4f-6ac198712243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:53.794250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26279163-7666-4fa3-8c4f-6ac198712243 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:53.794533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26279163-7666-4fa3-8c4f-6ac198712243 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:53.794865 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26279163-7666-4fa3-8c4f-6ac198712243 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:53.795955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26279163-7666-4fa3-8c4f-6ac198712243 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:53.797432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26279163-7666-4fa3-8c4f-6ac198712243 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:53.799233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 136/293] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:53 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:53.805295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:53.966698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:53.985373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb69db8-e510-4183-b1cd-223b175cd7a5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:54.039389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5b130-ed96-4235-8b26-cf6703efc09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:54.040002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b130-ed96-4235-8b26-cf6703efc09f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.040863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b130-ed96-4235-8b26-cf6703efc09f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.041136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b130-ed96-4235-8b26-cf6703efc09f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.041886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5b130-ed96-4235-8b26-cf6703efc09f None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=a775cb39facd44ca958b24d069b49eae) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:54.042193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5b130-ed96-4235-8b26-cf6703efc09f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:54.043629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5b130-ed96-4235-8b26-cf6703efc09f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:54.054487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 158/294] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:53 2022] GET /identity/v3/role_assignments?user.id=a775cb39facd44ca958b24d069b49eae&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:54.159458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05258d4e-b77a-4e26-bfe0-c837e1112d60 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.159999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05258d4e-b77a-4e26-bfe0-c837e1112d60 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.159999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05258d4e-b77a-4e26-bfe0-c837e1112d60 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.161365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 137/295] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:54.178627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40704557-d3ea-4461-8771-0d9f7c13d0db None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.178627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40704557-d3ea-4461-8771-0d9f7c13d0db None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.178627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40704557-d3ea-4461-8771-0d9f7c13d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.195958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-40704557-d3ea-4461-8771-0d9f7c13d0db None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:54.197579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-40704557-d3ea-4461-8771-0d9f7c13d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:54.285400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 159/296] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:54.551846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87963-f808-457c-ab9f-5ffdc3f0e8ba None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.552183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87963-f808-457c-ab9f-5ffdc3f0e8ba None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.552301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87963-f808-457c-ab9f-5ffdc3f0e8ba None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.553922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 138/297] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:54.559775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0063-6900-4e30-b7bf-78af98a289c9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.560278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0063-6900-4e30-b7bf-78af98a289c9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.560750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0063-6900-4e30-b7bf-78af98a289c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.580604 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d87963-f808-457c-ab9f-5ffdc3f0e8ba None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:54.582207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-230f0063-6900-4e30-b7bf-78af98a289c9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:54.583895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-230f0063-6900-4e30-b7bf-78af98a289c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:54.683173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 160/298] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:54.724920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d87963-f808-457c-ab9f-5ffdc3f0e8ba None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:54.765095 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3529d09-5ee2-4dd7-a940-505a31e051ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:54.765789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3529d09-5ee2-4dd7-a940-505a31e051ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.766060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3529d09-5ee2-4dd7-a940-505a31e051ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.766325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3529d09-5ee2-4dd7-a940-505a31e051ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.775761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:54.781781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3529d09-5ee2-4dd7-a940-505a31e051ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a3529d09-5ee2-4dd7-a940-505a31e051ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:54.786770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:54.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:43:54.789665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 139/299] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:54.794363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f0063-6900-4e30-b7bf-78af98a289c9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:54.809501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:54.865947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb237-116a-43f1-a8f1-8818670ddaac None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.866234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb237-116a-43f1-a8f1-8818670ddaac None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.866513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb237-116a-43f1-a8f1-8818670ddaac None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.868738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 140/300] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:54.873874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49634e-969d-42df-9a7a-ae568ff70850 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.873874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49634e-969d-42df-9a7a-ae568ff70850 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.874092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49634e-969d-42df-9a7a-ae568ff70850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.892887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5d49634e-969d-42df-9a7a-ae568ff70850 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:54.894099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5d49634e-969d-42df-9a7a-ae568ff70850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:54.910792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f0063-6900-4e30-b7bf-78af98a289c9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:54.952224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:54.952830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:54.953105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:54.953405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:54.954064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:54.954358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:54.955733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:54.964811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 161/301] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:54.965660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:54.970254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 141/302] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:43:54.971104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49634e-969d-42df-9a7a-ae568ff70850 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:55.043521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:55.069710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63ca51-d3c8-4d66-bcef-f4be2c16a6ab None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.087526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49634e-969d-42df-9a7a-ae568ff70850 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.105037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.105691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.105968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.106239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.116697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:55.120111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:55.121969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:55.123031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:55.123898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 162/303] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:55.128633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:55.129608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.130435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.130859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.131269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] PATH_INFO: `/v3/users/dae4351a1a85452c99a4e30821f80cd8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.132545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] RBAC: Authorizing `identity:get_user(user_id=dae4351a1a85452c99a4e30821f80cd8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:55.135781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:55.137838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 142/304] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:54 2022] GET /identity/v3/users/dae4351a1a85452c99a4e30821f80cd8 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:55.145334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:55.233759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f1e54-db40-43bd-97ea-14e52220eb42 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.267717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a830d8-255e-433b-85f7-15cec657ba13 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.273057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.273703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.273978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.274245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.275558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:55.275854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:55.277260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:55.279285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:55.287462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 163/305] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:55.290173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:55.305074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.305793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.306073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.306347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] PATH_INFO: `/v3/system/users/dae4351a1a85452c99a4e30821f80cd8/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.307746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dae4351a1a85452c99a4e30821f80cd8, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:55.309110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:55.319629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 143/306] 10.222.0.15 () {58 vars in 1334 bytes} [Fri Sep 9 14:43:55 2022] PUT /identity/v3/system/users/dae4351a1a85452c99a4e30821f80cd8/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:43:55.320491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:55.402900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:55.411490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed2582-a085-436d-ac9a-bcc74473a7f7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.452267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d991dbb-f514-4ed7-b5f9-e9a79199bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.452926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d991dbb-f514-4ed7-b5f9-e9a79199bd92 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.453236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d991dbb-f514-4ed7-b5f9-e9a79199bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.453517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d991dbb-f514-4ed7-b5f9-e9a79199bd92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.454678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d991dbb-f514-4ed7-b5f9-e9a79199bd92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:55.454895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb28cb-c601-4fc4-a4f1-053d3f0878f1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.456165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d991dbb-f514-4ed7-b5f9-e9a79199bd92 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:55.458184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 164/307] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:55.459124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55e5b6-c392-4001-984e-a0f026f88a80 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.459405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55e5b6-c392-4001-984e-a0f026f88a80 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.459676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55e5b6-c392-4001-984e-a0f026f88a80 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.461003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 165/308] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:55.466570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55e5b6-c392-4001-984e-a0f026f88a80 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:55.499617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bda2ea-e4ac-40fe-8c42-e194c676f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.500263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bda2ea-e4ac-40fe-8c42-e194c676f57d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.500575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bda2ea-e4ac-40fe-8c42-e194c676f57d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.501453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bda2ea-e4ac-40fe-8c42-e194c676f57d None admin] PATH_INFO: `/v3/users/560c561ce73f49cc8f0f6ce555be80c5` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.518925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bda2ea-e4ac-40fe-8c42-e194c676f57d None admin] RBAC: Authorizing `identity:get_user(user_id=560c561ce73f49cc8f0f6ce555be80c5)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:55.520407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bda2ea-e4ac-40fe-8c42-e194c676f57d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:55.522830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 144/309] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:55 2022] GET /identity/v3/users/560c561ce73f49cc8f0f6ce555be80c5 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:55.524852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1123d-cabe-4f2d-b6f4-cfb69e9260fb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.525131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1123d-cabe-4f2d-b6f4-cfb69e9260fb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.525478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1123d-cabe-4f2d-b6f4-cfb69e9260fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.543500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-28c1123d-cabe-4f2d-b6f4-cfb69e9260fb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:55.544689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-28c1123d-cabe-4f2d-b6f4-cfb69e9260fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:55.545616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:55.587290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55e5b6-c392-4001-984e-a0f026f88a80 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.620849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 145/310] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:55.622086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1123d-cabe-4f2d-b6f4-cfb69e9260fb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:55.630956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.631560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.631833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.632094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] PATH_INFO: `/v3/groups` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.632719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:55.634756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:55.653894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 166/311] 10.222.0.15 () {60 vars in 1134 bytes} [Fri Sep 9 14:43:55 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:55.704302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:55.766044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1123d-cabe-4f2d-b6f4-cfb69e9260fb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:55.806663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:55.807765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:55.808294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:55.808797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:55.809785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] RBAC: Adding query filter params (role.id=5b6d374901c24586a24d2483221bed64, user.id=560c561ce73f49cc8f0f6ce555be80c5, scope.system=all) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:55.810321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:55.812154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:55.818491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 146/312] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 14:43:55 2022] GET /identity/v3/role_assignments?user.id=560c561ce73f49cc8f0f6ce555be80c5&scope.system=all&role.id=5b6d374901c24586a24d2483221bed64 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:55.902199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:56.022294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301585a4-3727-4cdd-b498-516b0e83107a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:56.047775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:56.059849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:56.060484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:56.060765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:56.061041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:56.072465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:56.077853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:56.078765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:56.079547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 167/313] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:56.085431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:56.188686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02384032-d889-4c05-a39b-afa0f4bb796c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:56.204661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:56.225847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:56.226490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:56.226769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:56.227041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:56.227736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:56.228040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:56.229435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:56.237845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 147/314] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:56.244218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:56.350912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac8db3-1c93-4f03-8c79-7312444114b1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:56.384982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:56.385638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:56.385915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:56.386442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] PATH_INFO: `/v3/users/bcabe562523e46ed972fdc4c06f45731` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:56.387590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] RBAC: Authorizing `identity:get_user(user_id=bcabe562523e46ed972fdc4c06f45731)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:56.389026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:56.390945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 168/315] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:56 2022] GET /identity/v3/users/bcabe562523e46ed972fdc4c06f45731 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:56.396157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:56.503790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e49af-bb54-49b2-bed8-c4f4667c6abe None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:56.539893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:56.540568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:56.540850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:56.541127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:56.589525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:56.619024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:56.620798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:56.725798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 148/316] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:43:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:43:56.732202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:56.784211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:56.831237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdf074-8068-4fc5-984a-981dd081f80d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:56.868810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5fbd-a32c-44c6-8b74-c2ccf93cd30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:56.869462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5fbd-a32c-44c6-8b74-c2ccf93cd30c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:56.869736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5fbd-a32c-44c6-8b74-c2ccf93cd30c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:56.870005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5fbd-a32c-44c6-8b74-c2ccf93cd30c None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:56.871119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5fbd-a32c-44c6-8b74-c2ccf93cd30c None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:56.872552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5fbd-a32c-44c6-8b74-c2ccf93cd30c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:56.874534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 169/317] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:43:56 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:56.880085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:56.957222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:56.983000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ed83-5b4d-4df8-8843-9a4fe5a78de7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:57.025788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76854d-7265-4302-be2c-c145fe62f66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:57.026398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76854d-7265-4302-be2c-c145fe62f66c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:57.026682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76854d-7265-4302-be2c-c145fe62f66c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:57.026952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76854d-7265-4302-be2c-c145fe62f66c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:57.027682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76854d-7265-4302-be2c-c145fe62f66c None admin] RBAC: Adding query filter params (role.id=5b6d374901c24586a24d2483221bed64, scope.project.id=9b97fbead9af485db8541ad29cc7fe55, user.id=bcabe562523e46ed972fdc4c06f45731) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:57.027990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76854d-7265-4302-be2c-c145fe62f66c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:57.029439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76854d-7265-4302-be2c-c145fe62f66c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:57.035963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 149/318] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:43:56 2022] GET /identity/v3/role_assignments?user.id=bcabe562523e46ed972fdc4c06f45731&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=5b6d374901c24586a24d2483221bed64 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:43:57.605946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f517-c043-4cff-aaa3-d12ba315cabb None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:57.606463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f517-c043-4cff-aaa3-d12ba315cabb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:57.606463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f517-c043-4cff-aaa3-d12ba315cabb None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:57.607952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 170/319] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:57.613220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bae15-f30b-4dc3-93ef-6b2be8c3a1ab None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:57.613649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bae15-f30b-4dc3-93ef-6b2be8c3a1ab None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:57.613831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bae15-f30b-4dc3-93ef-6b2be8c3a1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:57.632139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e54bae15-f30b-4dc3-93ef-6b2be8c3a1ab None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:57.633379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e54bae15-f30b-4dc3-93ef-6b2be8c3a1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:57.700283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 150/320] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:57.967638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f517-c043-4cff-aaa3-d12ba315cabb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.006269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898dcc-b6e4-4559-ae29-b1f855743545 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.006580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898dcc-b6e4-4559-ae29-b1f855743545 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.007053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898dcc-b6e4-4559-ae29-b1f855743545 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.008396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 151/321] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:43:58.015288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e63e53-be6f-4870-80f9-0fa55761d4d2 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.015575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e63e53-be6f-4870-80f9-0fa55761d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.015847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e63e53-be6f-4870-80f9-0fa55761d4d2 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.017181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 152/322] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:43:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:43:58.018230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf9691-fe69-49a9-83d1-4d4c86678014 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.018597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf9691-fe69-49a9-83d1-4d4c86678014 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.018867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf9691-fe69-49a9-83d1-4d4c86678014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.037140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6abf9691-fe69-49a9-83d1-4d4c86678014 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:58.038340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6abf9691-fe69-49a9-83d1-4d4c86678014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:58.070230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f517-c043-4cff-aaa3-d12ba315cabb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.091880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:58.106221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.106849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.107124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.107533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.107667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:58.112300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 153/323] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:58.113511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bbba2-d02b-4f42-9d1c-d4a34e7c0259 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.113794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bbba2-d02b-4f42-9d1c-d4a34e7c0259 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.114065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bbba2-d02b-4f42-9d1c-d4a34e7c0259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.118830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:58.124646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:58.125711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:43:58.126516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:58.129486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 171/324] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:58.131375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.134326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ca2bbba2-d02b-4f42-9d1c-d4a34e7c0259 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:43:58.135505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ca2bbba2-d02b-4f42-9d1c-d4a34e7c0259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:43:58.209605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 154/325] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:43:58.239300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f44744-31cf-449a-a563-f30b93baee04 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.274621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.275520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.275967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.276407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.277250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:58.277717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:58.279312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:58.292777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 172/326] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:58.294395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bbba2-d02b-4f42-9d1c-d4a34e7c0259 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.377998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.396868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bbba2-d02b-4f42-9d1c-d4a34e7c0259 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.425217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:58.427158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:58.435049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.435994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.436272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.436541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] PATH_INFO: `/v3/users/dae4351a1a85452c99a4e30821f80cd8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.437699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] RBAC: Authorizing `identity:get_user(user_id=dae4351a1a85452c99a4e30821f80cd8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:58.439605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:58.441462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 155/327] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:58 2022] GET /identity/v3/users/dae4351a1a85452c99a4e30821f80cd8 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:58.447591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.490509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3f352-8578-4372-837f-343c4965c919 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.526091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.526718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.526996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.527257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.528351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:58.529814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:58.531352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 173/328] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:43:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:58.532182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.552644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d044-13b0-4bde-bbda-9c760d8958c5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.585720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.586321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.586596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.587364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.588079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] RBAC: Adding query filter params (role.id=72e8b68a7d364c4bb0fcf9e444949d58, user.id=dae4351a1a85452c99a4e30821f80cd8, scope.system=all) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:58.588380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:58.589853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:58.601129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 156/329] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 14:43:58 2022] GET /identity/v3/role_assignments?user.id=dae4351a1a85452c99a4e30821f80cd8&scope.system=all&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 510 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:58.601129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.634907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a4b2a-d1d1-4c13-81fa-49701bcfba9c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.646960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:58.675149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:58.677577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.678374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.679047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.679325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.692357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:43:58.698550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:43:58.699647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:43:58.702225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 174/330] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:43:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:43:58.705687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.735563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd6f4-2eb5-464d-8659-6c6940d04f5d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.777643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.778186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.778462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.778735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] PATH_INFO: `/v3/groups` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.779364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:58.780796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:58.799463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 157/331] 10.222.0.15 () {60 vars in 1134 bytes} [Fri Sep 9 14:43:58 2022] POST /identity/v3/groups => generated 226 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:43:58.820143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0794673-176b-409c-be8a-f88faa458c11 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:58.865675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:58.866313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:58.866591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:58.866862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:58.867549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:43:58.867851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:58.869346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:58.882849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 175/332] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:43:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:58.889389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:58.993749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaeb79-8028-4dd0-a998-787e99ec87d1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:59.010889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:59.036279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820dfab-7f86-405d-80e8-afa57a4ed489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:59.036852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820dfab-7f86-405d-80e8-afa57a4ed489 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:59.037136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820dfab-7f86-405d-80e8-afa57a4ed489 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:59.037469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820dfab-7f86-405d-80e8-afa57a4ed489 None admin] PATH_INFO: `/v3/users/560c561ce73f49cc8f0f6ce555be80c5` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:59.038613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820dfab-7f86-405d-80e8-afa57a4ed489 None admin] RBAC: Authorizing `identity:get_user(user_id=560c561ce73f49cc8f0f6ce555be80c5)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:59.040061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820dfab-7f86-405d-80e8-afa57a4ed489 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:59.041678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 158/333] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:43:58 2022] GET /identity/v3/users/560c561ce73f49cc8f0f6ce555be80c5 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:43:59.047339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:43:59.082324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:43:59.155221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be907-5499-458f-8832-2d8ea10130ac None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:43:59.189387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dffa0-2c95-47e6-843d-9fb16eb36002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:43:59.190037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dffa0-2c95-47e6-843d-9fb16eb36002 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:43:59.190314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dffa0-2c95-47e6-843d-9fb16eb36002 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:43:59.190590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dffa0-2c95-47e6-843d-9fb16eb36002 None admin] PATH_INFO: `/v3/system/users/560c561ce73f49cc8f0f6ce555be80c5/roles/5b6d374901c24586a24d2483221bed64` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:43:59.192039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dffa0-2c95-47e6-843d-9fb16eb36002 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=560c561ce73f49cc8f0f6ce555be80c5, role_id=5b6d374901c24586a24d2483221bed64)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:43:59.193464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dffa0-2c95-47e6-843d-9fb16eb36002 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:43:59.203634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 176/334] 10.222.0.15 () {58 vars in 1334 bytes} [Fri Sep 9 14:43:59 2022] PUT /identity/v3/system/users/560c561ce73f49cc8f0f6ce555be80c5/roles/5b6d374901c24586a24d2483221bed64 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:01.159130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24d384-ae89-45ac-9e90-620079925645 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:01.160106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24d384-ae89-45ac-9e90-620079925645 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:01.160632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24d384-ae89-45ac-9e90-620079925645 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:01.162492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 159/335] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:01.168846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d5f9a-bee9-486c-a121-23e0977202d7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:01.169720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d5f9a-bee9-486c-a121-23e0977202d7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:01.170184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d5f9a-bee9-486c-a121-23e0977202d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:01.194199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9c4d5f9a-bee9-486c-a121-23e0977202d7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:01.195551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9c4d5f9a-bee9-486c-a121-23e0977202d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:01.288080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 177/336] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:01.664613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062a087-3c86-436b-bcca-599a764ff491 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:01.665544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062a087-3c86-436b-bcca-599a764ff491 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:01.665795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062a087-3c86-436b-bcca-599a764ff491 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:01.667485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 160/337] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:01.672566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31dc15-798e-42de-818b-7b936fef2b39 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:01.672853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31dc15-798e-42de-818b-7b936fef2b39 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:01.673128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31dc15-798e-42de-818b-7b936fef2b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:01.703391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4d31dc15-798e-42de-818b-7b936fef2b39 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:01.704977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4d31dc15-798e-42de-818b-7b936fef2b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:01.728878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:01.750236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062a087-3c86-436b-bcca-599a764ff491 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:01.797862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 178/338] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:01.908551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062a087-3c86-436b-bcca-599a764ff491 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:01.942469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db327-3db2-48f1-9a67-ccfce3c4d37c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:01.943124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db327-3db2-48f1-9a67-ccfce3c4d37c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:01.943664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db327-3db2-48f1-9a67-ccfce3c4d37c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:01.946878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:01.947097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:01.947319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:01.947593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:01.948371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 179/339] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:01.953679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e19ad-1a9a-4404-8f9c-b2cf1542af2d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:01.954671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e19ad-1a9a-4404-8f9c-b2cf1542af2d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:01.955163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e19ad-1a9a-4404-8f9c-b2cf1542af2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:01.967826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:01.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:01.974033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:01.974807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 161/340] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:01.983358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:01.988190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6e7e19ad-1a9a-4404-8f9c-b2cf1542af2d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:01.989411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6e7e19ad-1a9a-4404-8f9c-b2cf1542af2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:02.052494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:02.123568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcd88c-b6fd-4308-95c7-efd97941aedc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:02.173433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 180/341] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:02.174980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e19ad-1a9a-4404-8f9c-b2cf1542af2d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:02.177509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:02.178721 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:02.179218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:02.179713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:02.180638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:02.181148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:02.182771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:02.197684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 162/342] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:44:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:02.207012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:02.326775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:02.362919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e19ad-1a9a-4404-8f9c-b2cf1542af2d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:02.402955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2b7c5-0705-4690-ac33-3341d420d8a8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:02.417535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:02.434709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eea38-cad7-4d23-bf3a-37eae075f0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:02.435359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eea38-cad7-4d23-bf3a-37eae075f0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:02.435677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eea38-cad7-4d23-bf3a-37eae075f0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:02.435898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eea38-cad7-4d23-bf3a-37eae075f0e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:02.453948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eea38-cad7-4d23-bf3a-37eae075f0e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-310eea38-cad7-4d23-bf3a-37eae075f0e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:02.458746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:02.459645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:02.460426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:02.461576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 181/343] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:44:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:02.466925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:02.467578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:02.467859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:02.468127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:02.469037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415ddf4-84c0-4799-bc2a-a5b91b5a9276 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:02.469331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415ddf4-84c0-4799-bc2a-a5b91b5a9276 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:02.469596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415ddf4-84c0-4799-bc2a-a5b91b5a9276 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:02.470899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 182/344] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:02.471584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415ddf4-84c0-4799-bc2a-a5b91b5a9276 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:02.594426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:02.595922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:02.651585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415ddf4-84c0-4799-bc2a-a5b91b5a9276 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:02.701428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec76fd-1c52-45a4-bcba-9915459345e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:02.701428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec76fd-1c52-45a4-bcba-9915459345e4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:02.701428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec76fd-1c52-45a4-bcba-9915459345e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:02.701428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec76fd-1c52-45a4-bcba-9915459345e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:02.701823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec76fd-1c52-45a4-bcba-9915459345e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:02.701823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec76fd-1c52-45a4-bcba-9915459345e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:02.701823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec76fd-1c52-45a4-bcba-9915459345e4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:02.713884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 183/345] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:02.715233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0366f1-efbc-4d84-b685-80952d4441d5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:02.715521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0366f1-efbc-4d84-b685-80952d4441d5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:02.715798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0366f1-efbc-4d84-b685-80952d4441d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:02.741788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4b0366f1-efbc-4d84-b685-80952d4441d5 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:02.743318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4b0366f1-efbc-4d84-b685-80952d4441d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:02.743433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 163/346] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 14:44:02.744116 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:02.745123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:02.754706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:02.826626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 184/347] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:02.834284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0366f1-efbc-4d84-b685-80952d4441d5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:02.872206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9e47-9c25-49ad-aed6-bf12c7022974 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:02.920815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:02.921480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:02.921758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:02.922028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:02.943718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:02.954140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:02.955101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 14:44:02.955894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:02.956498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 164/348] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:02.957521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.023956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0366f1-efbc-4d84-b685-80952d4441d5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.100547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ae469-58ac-4a64-a651-03cfb9fa6d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.101178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae469-58ac-4a64-a651-03cfb9fa6d1e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.101481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae469-58ac-4a64-a651-03cfb9fa6d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.101752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae469-58ac-4a64-a651-03cfb9fa6d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.106157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:03.121884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:03.156080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c1bc-f342-4940-be8f-7135cc180fe1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.194818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.195439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.195719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.196016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.197163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:03.199187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:03.200923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 165/349] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:02 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:03.201712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.215012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ae469-58ac-4a64-a651-03cfb9fa6d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:03.217066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ae469-58ac-4a64-a651-03cfb9fa6d1e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:03.330544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0f7e-1820-4cd4-a51e-b1fc4bca4047 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.343083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 185/350] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:03.344149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640501-b402-4594-b331-1d889348de50 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.344421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640501-b402-4594-b331-1d889348de50 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.344687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640501-b402-4594-b331-1d889348de50 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.346331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 186/351] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:03.351448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640501-b402-4594-b331-1d889348de50 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.374267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.374874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.375149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.375421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.376080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:03.376377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:03.377808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:03.379574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:03.390827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 166/352] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:03.390827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.416625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:03.490460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640501-b402-4594-b331-1d889348de50 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.514874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b93ba-a98a-4fb5-9e22-e3a820b91f6a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.532430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.533324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.533606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.533894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.546385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:03.547871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:03.549788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 187/353] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:03.550388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.560969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46f0b3-59c6-462f-99e9-cc57c8069718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.562024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f0b3-59c6-462f-99e9-cc57c8069718 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.562481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f0b3-59c6-462f-99e9-cc57c8069718 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.562927 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f0b3-59c6-462f-99e9-cc57c8069718 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.576243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46f0b3-59c6-462f-99e9-cc57c8069718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ca46f0b3-59c6-462f-99e9-cc57c8069718 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:03.582946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:03.583839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:03.584617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:03.586634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 167/354] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:03.591595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b925-e9f8-4431-84d5-861382b9f556 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.591595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b925-e9f8-4431-84d5-861382b9f556 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.591595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b925-e9f8-4431-84d5-861382b9f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.616305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cbd2b925-e9f8-4431-84d5-861382b9f556 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:03.618071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cbd2b925-e9f8-4431-84d5-861382b9f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:03.689545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:03.691723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7df4b-b857-48df-9474-9ad37742c272 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.695862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:03.710285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 168/355] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:03.711461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b925-e9f8-4431-84d5-861382b9f556 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.738909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.739580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.740083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.740365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.742007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:03.744247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:03.747961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 188/356] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:03.749015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.831528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b925-e9f8-4431-84d5-861382b9f556 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.868805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcc1b2-2857-46e2-a637-f4cc96609210 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:03.882499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.884361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.884637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.885323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] PATH_INFO: `/v3/users/560c561ce73f49cc8f0f6ce555be80c5` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.886953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] RBAC: Authorizing `identity:get_user(user_id=560c561ce73f49cc8f0f6ce555be80c5)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:03.888856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:03.890963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 169/357] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/users/560c561ce73f49cc8f0f6ce555be80c5 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:03.892222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.911962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:03.912568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:03.912844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:03.913117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:03.913969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:03.914276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:03.915755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:03.922017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 189/358] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:03.923127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:03.963420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:03.970839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:04.036448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04285c-a0bd-45e3-bba2-18d942564f07 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.060370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85128a0-0292-4349-8187-9e14a2c521c0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.081814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeed86-8cb4-4c37-9313-9a5298409054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.082555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeed86-8cb4-4c37-9313-9a5298409054 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.082891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeed86-8cb4-4c37-9313-9a5298409054 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.083232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeed86-8cb4-4c37-9313-9a5298409054 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.083984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeed86-8cb4-4c37-9313-9a5298409054 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:04.084349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeed86-8cb4-4c37-9313-9a5298409054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:04.085991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeed86-8cb4-4c37-9313-9a5298409054 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:04.101142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.101933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.102284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.102631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.102843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 170/359] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:04.104202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94690-e54d-4b26-b3bc-ea51f17416dd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.104501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94690-e54d-4b26-b3bc-ea51f17416dd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.104754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94690-e54d-4b26-b3bc-ea51f17416dd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.106251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 171/360] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:04.106781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec94690-e54d-4b26-b3bc-ea51f17416dd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.108901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:04.110423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:04.112356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 190/361] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:03 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:04.113252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.242988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9952e5-9fb4-466f-a1fb-9400e0710769 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.245163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:04.255639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec94690-e54d-4b26-b3bc-ea51f17416dd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.277340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:04.287488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.288130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.288410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.288676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.299867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7655848-8520-452e-a919-0f5025d8c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.300492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7655848-8520-452e-a919-0f5025d8c916 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.300756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7655848-8520-452e-a919-0f5025d8c916 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.300972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:04.301093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7655848-8520-452e-a919-0f5025d8c916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.301804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7655848-8520-452e-a919-0f5025d8c916 None admin] RBAC: Adding query filter params (role.id=5b6d374901c24586a24d2483221bed64, user.id=560c561ce73f49cc8f0f6ce555be80c5, scope.system=all) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:04.302108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7655848-8520-452e-a919-0f5025d8c916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:04.303526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7655848-8520-452e-a919-0f5025d8c916 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:04.306890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:04.307765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:04.308600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 191/362] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:04.309317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.325762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 172/363] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/role_assignments?user.id=560c561ce73f49cc8f0f6ce555be80c5&scope.system=all&role.id=5b6d374901c24586a24d2483221bed64 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:04.327074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f7dd-1d82-4af7-bf81-7b1728a18fed None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.327358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f7dd-1d82-4af7-bf81-7b1728a18fed None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.327631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f7dd-1d82-4af7-bf81-7b1728a18fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.347241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-82d1f7dd-1d82-4af7-bf81-7b1728a18fed None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:04.348487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-82d1f7dd-1d82-4af7-bf81-7b1728a18fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:04.435770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4a06f-7c7f-46ad-81a9-1d9b0d83a45b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.445534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 173/364] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:04.450945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1f7dd-1d82-4af7-bf81-7b1728a18fed None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.493754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.494299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.494576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.494847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.541629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:04.609842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:04.611986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:04.613662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1f7dd-1d82-4af7-bf81-7b1728a18fed None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.616528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:04.671881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.672839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.673145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.673446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.674487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:04.674796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:04.676234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:04.687458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 174/365] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:04.688294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.763768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 192/366] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:04.769486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.858062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:04.890714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c7b28-86db-48ab-b7e8-ee831167c00b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.916850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd32d2b-e434-4b29-9103-48a41824667a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:04.938381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.938988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.939264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.939530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.940190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:04.940487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:04.941958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:04.951634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 175/367] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:04.952458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.962878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:04.963208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:04.963483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:04.963750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:04.964996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:04.966599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:04.968399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 193/368] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:04.969109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:04.994205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:05.101365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ac2fd-c5ec-4fea-b081-29058c30353f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.106970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767f03-b964-42ef-ad2e-6010844cff06 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.134567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:05.141869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.143044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.143333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.143610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.147993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.148890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.149981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.150460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.163704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:05.169790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:05.170663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:05.171465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:05.174672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 176/369] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:05.175847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:05.224093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:05.226019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:05.301731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:05.315215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25683c2-847c-4ec5-8a7b-94d8b30fe06d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.362694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 194/370] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:05.364201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:05.370356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.371022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.371298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.371573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.373751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:05.375218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:05.376827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 177/371] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:05 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:05.381225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:05.417133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:05.511713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be3c93-dfb0-41ab-8701-ce0a0660fae7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.522128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fb878-675a-49c0-a462-3a6f9b17145d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.549748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.550601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.551113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.551608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.552553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:05.553069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:05.554677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:05.562261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.563960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.563960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.563960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.564567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:05.566158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:05.567056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 195/372] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:05.567794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 178/373] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:05 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:05.568474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:05.572772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:05.620424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:05.669187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:05.679591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37284-05ed-480f-93c4-d3508039e68d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.686292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d391-bbb2-40e3-a091-1e73c9ebeb08 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.716634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.717273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.717559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.717834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.718560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=9b97fbead9af485db8541ad29cc7fe55) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:05.718868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:05.720315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:05.727938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 196/374] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:05 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:05.728107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.728803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.729020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:05.729333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.729674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.853219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005606e9-7566-4e3b-8ade-7b05f1567d6c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:05.863678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:05.865305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:05.890760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:05.891519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:05.891809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:05.892090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:05.893274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:05.894750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:05.896504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 197/375] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:05 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:05.904675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:05.905250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:05.945813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:06.007606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 179/376] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:06.015484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:06.050115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4b9f7-22c8-4145-bb54-2231af03781a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:06.090956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:06.091829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:06.092264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:06.092548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:06.093309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=9b97fbead9af485db8541ad29cc7fe55) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:06.093623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:06.095522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:06.103468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 198/377] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:05 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:06.173707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c4d2f-eef4-4b8e-85f4-a2474a0531f1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:06.212187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:06.222267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:06.222905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:06.223178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:06.223448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] PATH_INFO: `/v3/projects/5aff011fa0544761b3d08887135e99aa` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:06.230077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] RBAC: Authorizing `identity:get_project(project_id=5aff011fa0544761b3d08887135e99aa)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:06.231547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:06.233474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 180/378] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:06 2022] GET /identity/v3/projects/5aff011fa0544761b3d08887135e99aa => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:06.239765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:06.377265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:06.397293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09425d1-0537-417d-a34d-8572b75f6449 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:06.440343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b904093-98ce-4c8d-82a4-5b0a60d7e784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:06.440984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904093-98ce-4c8d-82a4-5b0a60d7e784 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:06.441286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904093-98ce-4c8d-82a4-5b0a60d7e784 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:06.441561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904093-98ce-4c8d-82a4-5b0a60d7e784 None admin] PATH_INFO: `/v3/groups/aa14a55146c041eb919c1ed610d5d5a2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:06.450819 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b904093-98ce-4c8d-82a4-5b0a60d7e784 None admin] RBAC: Authorizing `identity:get_group(group_id=aa14a55146c041eb919c1ed610d5d5a2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:06.452347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b904093-98ce-4c8d-82a4-5b0a60d7e784 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:06.454067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 199/379] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:06 2022] GET /identity/v3/groups/aa14a55146c041eb919c1ed610d5d5a2 => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:06.460138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:06.582326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6468010-e735-41ec-a550-d2c31071da42 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:06.618745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2e63f-1364-4615-a7c8-81a02fcf82e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:06.619364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2e63f-1364-4615-a7c8-81a02fcf82e3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:06.619640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2e63f-1364-4615-a7c8-81a02fcf82e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:06.619905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2e63f-1364-4615-a7c8-81a02fcf82e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:06.620633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2e63f-1364-4615-a7c8-81a02fcf82e3 None admin] RBAC: Adding query filter params (group.id=aa14a55146c041eb919c1ed610d5d5a2, role.id=82245735988747a4bc5704727a1cf374, scope.project.id=5aff011fa0544761b3d08887135e99aa) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:06.620938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2e63f-1364-4615-a7c8-81a02fcf82e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:06.622407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2e63f-1364-4615-a7c8-81a02fcf82e3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:06.632328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 181/380] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:06 2022] GET /identity/v3/role_assignments?group.id=aa14a55146c041eb919c1ed610d5d5a2&scope.project.id=5aff011fa0544761b3d08887135e99aa&role.id=82245735988747a4bc5704727a1cf374 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:06.663054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563655-3dc4-4e8f-a7a7-43d55393b783 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:06.663374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563655-3dc4-4e8f-a7a7-43d55393b783 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:06.663603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563655-3dc4-4e8f-a7a7-43d55393b783 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:06.665256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 200/381] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:06.671814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b9c7-58ec-4927-a707-93c5fcb3c8f1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:06.672096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b9c7-58ec-4927-a707-93c5fcb3c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:06.672371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b9c7-58ec-4927-a707-93c5fcb3c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:06.691584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1719b9c7-58ec-4927-a707-93c5fcb3c8f1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:06.692799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1719b9c7-58ec-4927-a707-93c5fcb3c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:06.766317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:06.784653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 182/382] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:07.113966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc563655-3dc4-4e8f-a7a7-43d55393b783 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:07.229715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc563655-3dc4-4e8f-a7a7-43d55393b783 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:07.266430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:07.266984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:07.267260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:07.267526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:07.278463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:07.283474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:07.284389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:07.285676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 201/383] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:07.291284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719b9c7-58ec-4927-a707-93c5fcb3c8f1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:07.398030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719b9c7-58ec-4927-a707-93c5fcb3c8f1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:07.435377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:07.435996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:07.436276 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:07.436600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:07.437311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:07.437657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:07.439047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:07.449782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 183/384] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:44:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:07.455276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:07.483522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:07.589908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9a8cb-9fd7-4a69-bd45-50515ed7795a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:07.637662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:07.638319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:07.638602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:07.638877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:07.640060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:07.641586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:07.643593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 202/385] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:07 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:07.649149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:07.733676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:07.796109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce517020-1fc8-49ca-b46d-4cca51f8caa5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:07.844932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6825c78-1caa-4f6a-bb06-2aa174ae8b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:07.845968 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6825c78-1caa-4f6a-bb06-2aa174ae8b56 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:07.846530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6825c78-1caa-4f6a-bb06-2aa174ae8b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:07.847009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6825c78-1caa-4f6a-bb06-2aa174ae8b56 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:07.853304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6825c78-1caa-4f6a-bb06-2aa174ae8b56 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:07.854992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6825c78-1caa-4f6a-bb06-2aa174ae8b56 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:07.857318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 184/386] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:07 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:07.866245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:07.935711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786453f-db19-4b58-bdde-7be4523eb717 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:07.936027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786453f-db19-4b58-bdde-7be4523eb717 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:07.936257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786453f-db19-4b58-bdde-7be4523eb717 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:07.937926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 185/387] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:07.946744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282bb3-273d-40cc-83e9-2cca4cc8bca1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:07.946744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282bb3-273d-40cc-83e9-2cca4cc8bca1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:07.946744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282bb3-273d-40cc-83e9-2cca4cc8bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:07.980886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-63282bb3-273d-40cc-83e9-2cca4cc8bca1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:07.982124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-63282bb3-273d-40cc-83e9-2cca4cc8bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:07.997840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b5aa-2c59-411c-b340-17ba8f3d80b6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:08.001135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:08.075226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:08.076178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:08.076640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:08.078521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/groups/78d06f724ade4ce4be3a0587c90bf520/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:08.084050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4eabd47bff3c4b6eb0e462a0797efb23, group_id=78d06f724ade4ce4be3a0587c90bf520, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:08.085897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:08.111742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 203/388] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 14:44:07 2022] PUT /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/groups/78d06f724ade4ce4be3a0587c90bf520/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:08.127985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 186/389] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:08.428501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:08.524696 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030b64d-ccef-45ad-abd8-5dec827f13c4 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:08.524978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030b64d-ccef-45ad-abd8-5dec827f13c4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:08.525273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030b64d-ccef-45ad-abd8-5dec827f13c4 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:08.526928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 187/390] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:08.536484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94248ad-f555-426b-8a03-45a5e01ae8db None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:08.536959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94248ad-f555-426b-8a03-45a5e01ae8db None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:08.537403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94248ad-f555-426b-8a03-45a5e01ae8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:08.544732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2eda7-24bd-410d-af89-f28a06ec09b2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:08.569952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e94248ad-f555-426b-8a03-45a5e01ae8db None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:08.576724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e94248ad-f555-426b-8a03-45a5e01ae8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:08.612462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:08.614733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:08.615245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:08.616641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:08.634782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:08.643743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:08.651698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:08.652828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:08.653849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:08.666714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 204/391] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:44:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:08.671193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:08.673701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:08.731745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 188/392] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:08.825252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05bd0b-2e89-4f32-a697-6048bfb217f1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:08.825252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05bd0b-2e89-4f32-a697-6048bfb217f1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:08.825546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05bd0b-2e89-4f32-a697-6048bfb217f1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:08.826902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 189/393] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:08.831782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb585f-234d-4618-8863-ad7e9578c808 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:08.832064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb585f-234d-4618-8863-ad7e9578c808 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:08.832335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb585f-234d-4618-8863-ad7e9578c808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:08.844950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a49d5-e8e4-449c-acd7-d2214ca251e8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:08.852506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9afb585f-234d-4618-8863-ad7e9578c808 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:08.853928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9afb585f-234d-4618-8863-ad7e9578c808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:08.911126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:08.911744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:08.912020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:08.912292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:08.912963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:08.913285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:08.914794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:08.923857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 205/394] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:08.934264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:08.965629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 190/395] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:09.035607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:09.046213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb585f-234d-4618-8863-ad7e9578c808 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.114334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331b5f3-bbe6-41e1-b9b8-06f656b97964 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.149826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.150513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.150792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.151094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.152220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:09.153718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:09.155606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 206/396] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:08 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:09.160957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.264177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1531c-95d8-4128-ab95-e7fcda67ff5f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.279871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb585f-234d-4618-8863-ad7e9578c808 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.296962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:09.298187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:09.304212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.305018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.306315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.306836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.308194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:09.309826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:09.311778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 207/397] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:09.317629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.350016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.350710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.350985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.351259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.362075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:09.366934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:09.368560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:09.369478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 191/398] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:09.370139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.435706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80390529-2ada-44fc-b1ee-9be6be629ee4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.474314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.475441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.476184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.476765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/groups/78d06f724ade4ce4be3a0587c90bf520/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.480476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] RBAC: Authorizing `identity:create_grant(project_id=4eabd47bff3c4b6eb0e462a0797efb23, group_id=78d06f724ade4ce4be3a0587c90bf520, role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:09.482608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:09.485184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56681a-b087-4445-8c08-3348a918809e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.500737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 208/399] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 14:44:09 2022] PUT /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23/groups/78d06f724ade4ce4be3a0587c90bf520/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:44:09.501815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.543954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffac71-ffe9-4ab3-9f2f-24d2e6c1249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.544712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffac71-ffe9-4ab3-9f2f-24d2e6c1249f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.544994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffac71-ffe9-4ab3-9f2f-24d2e6c1249f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.545301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffac71-ffe9-4ab3-9f2f-24d2e6c1249f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.560449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffac71-ffe9-4ab3-9f2f-24d2e6c1249f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d3ffac71-ffe9-4ab3-9f2f-24d2e6c1249f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:09.566395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:09.567547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:09.568432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 192/400] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:09.574263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6a364-5492-47fb-8d91-afa25932e708 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.574556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6a364-5492-47fb-8d91-afa25932e708 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.574834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6a364-5492-47fb-8d91-afa25932e708 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.577113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 193/401] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:09.577896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6a364-5492-47fb-8d91-afa25932e708 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.580210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:09.616218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:09.641890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de668588-27be-4c9c-84ee-a222f464695e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.688453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf9e37-8a81-4d6e-b625-29c6d5615400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.690007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9e37-8a81-4d6e-b625-29c6d5615400 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.690496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9e37-8a81-4d6e-b625-29c6d5615400 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.691255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9e37-8a81-4d6e-b625-29c6d5615400 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.692143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf9e37-8a81-4d6e-b625-29c6d5615400 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:09.692795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf9e37-8a81-4d6e-b625-29c6d5615400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:09.694616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf9e37-8a81-4d6e-b625-29c6d5615400 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:09.707251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 209/402] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:09.708406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e96be1-23cc-4690-8b3d-1406a52b2fdf None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.708693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e96be1-23cc-4690-8b3d-1406a52b2fdf None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.708965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e96be1-23cc-4690-8b3d-1406a52b2fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.710570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6a364-5492-47fb-8d91-afa25932e708 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.727801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f0e96be1-23cc-4690-8b3d-1406a52b2fdf None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:09.728986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f0e96be1-23cc-4690-8b3d-1406a52b2fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:09.753353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.754055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.754396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.754731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.756059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:09.756698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:09.758446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:09.768177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 194/403] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:09.768904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.823782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 210/404] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:09.824660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e96be1-23cc-4690-8b3d-1406a52b2fdf None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:09.881692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:09.910920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:09.932047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb4979-5fda-4d0c-9a1c-a5b3415f9813 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.943860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e96be1-23cc-4690-8b3d-1406a52b2fdf None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:09.983692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:09.984324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:09.984604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:09.985005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:09.986423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:09.988290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:09.990164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 211/405] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:09.995695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:10.008551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.025835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.026125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.026399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.027649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:10.029087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:10.031692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 195/406] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:10.037120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:10.149944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09ae01-d0de-41c9-a57e-84c9b93b2f26 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:10.173433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:10.194152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.194833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.195112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.195385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.196597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:10.198438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:10.200337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 212/407] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:09 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:10.206684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:10.250606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae352ee-1029-4985-8a98-a15deff74db5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:10.287920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.288831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.289112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.289422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.291257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:10.292840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:10.294839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 196/408] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:10 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:10.295620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:10.322642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557299ef-3b66-40ec-a64b-ad1577e689b1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:10.328192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:10.359560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.360222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.360541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.360815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55/groups/78d06f724ade4ce4be3a0587c90bf520/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.363208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] RBAC: Authorizing `identity:create_grant(project_id=9b97fbead9af485db8541ad29cc7fe55, group_id=78d06f724ade4ce4be3a0587c90bf520, role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:10.364806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:10.381164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 213/409] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 14:44:10 2022] PUT /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55/groups/78d06f724ade4ce4be3a0587c90bf520/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:10.382160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:10.451688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b398ba-05e9-45c8-a5c2-86cfe3c17407 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:10.460466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:10.527917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294376b1-2909-4c8c-a560-a00f5313629f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:10.546014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.546666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.546940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.547205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.572064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:10.583370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:10.584333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:10.585140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:10.590037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 197/410] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:10.598798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:10.605425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.606102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.606425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.606703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55/groups/78d06f724ade4ce4be3a0587c90bf520/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.613233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] RBAC: Authorizing `identity:create_grant(project_id=9b97fbead9af485db8541ad29cc7fe55, group_id=78d06f724ade4ce4be3a0587c90bf520, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:10.614698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:10.634998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 214/411] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 14:44:10 2022] PUT /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55/groups/78d06f724ade4ce4be3a0587c90bf520/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:10.725468 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:10.744318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c7dbf-3e83-4b9e-acac-d8ac6e4b18db None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:10.780709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8cd1d-91c3-477f-a4d7-bb876c878ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.781367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8cd1d-91c3-477f-a4d7-bb876c878ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.781651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8cd1d-91c3-477f-a4d7-bb876c878ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.781921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8cd1d-91c3-477f-a4d7-bb876c878ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.782890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8cd1d-91c3-477f-a4d7-bb876c878ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:10.783189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8cd1d-91c3-477f-a4d7-bb876c878ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:10.784782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8cd1d-91c3-477f-a4d7-bb876c878ab8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:10.799497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 198/412] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:10.802274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:10.814731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c09a1-b656-4b9e-a516-cf218f90e212 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.815034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c09a1-b656-4b9e-a516-cf218f90e212 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.815289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c09a1-b656-4b9e-a516-cf218f90e212 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.816986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 199/413] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:10.827419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ae2f-ab96-454b-9a10-69dee9927f37 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.827419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ae2f-ab96-454b-9a10-69dee9927f37 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.827697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ae2f-ab96-454b-9a10-69dee9927f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.854120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2c28ae2f-ab96-454b-9a10-69dee9927f37 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:10.855316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2c28ae2f-ab96-454b-9a10-69dee9927f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:10.936077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 200/414] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:10.954109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fac241-7026-42ae-98c3-3506b82f89ee None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:10.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:10.991406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb758-580d-439e-b223-4477826933af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:10.992358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb758-580d-439e-b223-4477826933af None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:10.992885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb758-580d-439e-b223-4477826933af None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:10.993356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb758-580d-439e-b223-4477826933af None admin] PATH_INFO: `/v3/groups/aa14a55146c041eb919c1ed610d5d5a2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:10.994640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb758-580d-439e-b223-4477826933af None admin] RBAC: Authorizing `identity:get_group(group_id=aa14a55146c041eb919c1ed610d5d5a2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:10.996314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb758-580d-439e-b223-4477826933af None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:10.998430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 215/415] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:10 2022] GET /identity/v3/groups/aa14a55146c041eb919c1ed610d5d5a2 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:11.006407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28ae2f-ab96-454b-9a10-69dee9927f37 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:11.114622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:11.138285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28ae2f-ab96-454b-9a10-69dee9927f37 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:11.173899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:11.174542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:11.174820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:11.175090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] PATH_INFO: `/v3/projects/5aff011fa0544761b3d08887135e99aa` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:11.176206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] RBAC: Authorizing `identity:get_project(project_id=5aff011fa0544761b3d08887135e99aa)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:11.177670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:11.179607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 201/416] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:11 2022] GET /identity/v3/projects/5aff011fa0544761b3d08887135e99aa => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:11.185321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb758-580d-439e-b223-4477826933af None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:11.239154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:11.346188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb758-580d-439e-b223-4477826933af None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:11.405915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:11.406733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:11.407106 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:11.407469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] PATH_INFO: `/v3/projects/5aff011fa0544761b3d08887135e99aa/groups/aa14a55146c041eb919c1ed610d5d5a2/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:11.410831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] RBAC: Authorizing `identity:create_grant(project_id=5aff011fa0544761b3d08887135e99aa, group_id=aa14a55146c041eb919c1ed610d5d5a2, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:11.413756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:11.431776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d29047-f535-45b1-9737-3ed041ac7c88 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:11.435193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 216/417] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 14:44:11 2022] PUT /identity/v3/projects/5aff011fa0544761b3d08887135e99aa/groups/aa14a55146c041eb919c1ed610d5d5a2/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:11.475708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:11.476413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:11.476688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:11.476956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:11.488157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:11.493946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:11.494954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:11.496528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 202/418] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:11.501136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:11.551122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:11.639619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d4658-8aa6-4912-8315-123e1b453f42 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:11.681789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5dcd7-a308-448a-bbfc-de17a97c9e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:11.682763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5dcd7-a308-448a-bbfc-de17a97c9e24 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:11.683280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5dcd7-a308-448a-bbfc-de17a97c9e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:11.683749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5dcd7-a308-448a-bbfc-de17a97c9e24 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:11.684674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5dcd7-a308-448a-bbfc-de17a97c9e24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:11.685163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5dcd7-a308-448a-bbfc-de17a97c9e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:11.687010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5dcd7-a308-448a-bbfc-de17a97c9e24 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:11.698988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 217/419] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:44:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:11.704385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:11.719155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:11.841248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06067-43cd-4e5d-9ee2-39f0f3e9d7c5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:11.888985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:11.889674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:11.889899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:11.890207 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:11.982792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:11.984237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:11.992467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:12.071215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434bec9-ba00-475c-ab17-04e629f4670c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:12.071956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434bec9-ba00-475c-ab17-04e629f4670c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:12.072471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434bec9-ba00-475c-ab17-04e629f4670c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:12.075151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 218/420] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:12.080180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a03f0-c60f-446f-a87b-32c13047fdfb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:12.080746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a03f0-c60f-446f-a87b-32c13047fdfb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:12.081262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a03f0-c60f-446f-a87b-32c13047fdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:12.118363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3d3a03f0-c60f-446f-a87b-32c13047fdfb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:12.119679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3d3a03f0-c60f-446f-a87b-32c13047fdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:12.125532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 203/421] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:12.131781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:12.245099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 219/422] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 09 14:44:12.309324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fae20-ead7-4070-9bb9-ece315c20381 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:12.319423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:12.351816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:12.352547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:12.352825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:12.353090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:12.354236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:12.355652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:12.357691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 204/423] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:12 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:12.366462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a03f0-c60f-446f-a87b-32c13047fdfb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:12.412426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:12.515389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a03f0-c60f-446f-a87b-32c13047fdfb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:12.558337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:12.559034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:12.559317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:12.559589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:12.560707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:12.562175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:12.564030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 220/424] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:12 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:12.571850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:12.627064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:12.738558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6506-d88c-40c5-a181-f10a5c698d2c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:12.779663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:12.780659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:12.781139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:12.781638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:12.781814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:12.783219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:12.783750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:12.785827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:12.793051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 205/425] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:12 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:12.797439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd7c4b-4609-4e1a-a0f1-c9e172f299c6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:12.865092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:12.865783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:12.866059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:12.866326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:12.884200 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:12.889944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:12.890892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:12.891679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:12.897326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 221/426] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:44:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:12.898964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:12.973401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:13.077811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbbeb3-dd60-4994-88a3-e23a8aeb2a64 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:13.120978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c8a52-c44b-49f1-af3e-c1e2b5b7f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:13.121645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c8a52-c44b-49f1-af3e-c1e2b5b7f068 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.121923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c8a52-c44b-49f1-af3e-c1e2b5b7f068 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.122190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c8a52-c44b-49f1-af3e-c1e2b5b7f068 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.122873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c8a52-c44b-49f1-af3e-c1e2b5b7f068 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:13.123173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c8a52-c44b-49f1-af3e-c1e2b5b7f068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:13.124595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c8a52-c44b-49f1-af3e-c1e2b5b7f068 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:13.139463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 206/427] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:13.145568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:13.255800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f84de5-c464-4255-b94e-f68a366499b0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.256669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f84de5-c464-4255-b94e-f68a366499b0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.257168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f84de5-c464-4255-b94e-f68a366499b0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.259054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 207/428] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:13.267437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1eff20-bdc7-4547-88c8-27c22119450f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.267991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1eff20-bdc7-4547-88c8-27c22119450f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.268489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1eff20-bdc7-4547-88c8-27c22119450f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.278775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e176866-8aaf-4505-a161-29918f6ccd12 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:13.293464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5c1eff20-bdc7-4547-88c8-27c22119450f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:13.294663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5c1eff20-bdc7-4547-88c8-27c22119450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:13.319314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:13.320258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.320531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.320865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.353228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:13.386848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 208/429] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:13.389534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6370471-68dc-440b-9ef6-6ecbee2c7b76 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.389534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6370471-68dc-440b-9ef6-6ecbee2c7b76 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.389534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6370471-68dc-440b-9ef6-6ecbee2c7b76 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.393849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 209/430] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:13.399370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d873a-d95b-43bb-a386-a4329faf1231 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.399765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d873a-d95b-43bb-a386-a4329faf1231 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.400142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d873a-d95b-43bb-a386-a4329faf1231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.410374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:13.412087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:13.419412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2d2d873a-d95b-43bb-a386-a4329faf1231 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:13.420697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2d2d873a-d95b-43bb-a386-a4329faf1231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:13.446043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:13.492238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 210/431] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:13.531215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 222/432] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:13.536779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d873a-d95b-43bb-a386-a4329faf1231 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:13.650736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d873a-d95b-43bb-a386-a4329faf1231 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:13.687524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:13.690378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:13.691690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.691690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.691690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] PATH_INFO: `/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.692703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] RBAC: Authorizing `identity:get_project(project_id=4eabd47bff3c4b6eb0e462a0797efb23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:13.694188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:13.696127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 211/433] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:13 2022] GET /identity/v3/projects/4eabd47bff3c4b6eb0e462a0797efb23 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:13.701488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:13.731388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:13.759496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:13.808366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019505a-ace9-4de4-b898-bd64576e64fb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:13.809496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec28fc2-5729-44e4-98c3-826210e903b4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:13.845100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:13.845766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.846043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.846319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.847423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:13.847973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:13.848600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.848849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:13.848964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:13.849140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:13.850660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 212/434] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:13 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:13.854108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:13.862088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:13.867001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:13.868411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:13.869785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:13.873490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 223/435] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:44:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:13.874132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:13.954728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:13.960404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b83239-6c99-4a50-b7ab-d4a3b37271a2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:13.984285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d51c0-780c-4a5d-8c91-9b3f9b8112ae None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:13.998813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:13.999512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:13.999715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.000162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.001064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.014407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:14.019945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:14.020899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:14.022141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 213/436] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:14.022812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.029013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.029659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.029942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.030213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.030942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:14.031249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.032860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.038934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 224/437] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:13 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=4eabd47bff3c4b6eb0e462a0797efb23&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:14.040252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.146530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9c07-0cfe-4c16-99fd-b40c120bbe2e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.173492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c27af-917a-4c01-9f90-de03decd55b5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.187675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.188027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.188363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.188644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.189498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:14.189695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.191116 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.200819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 214/438] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:14.206550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.217004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.217994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.218570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.219050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.220012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:14.220504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.222134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.227830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.235441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 225/439] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:14.241809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.290211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.334429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b8927-e4c4-4dbe-b353-78da713b87a3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.356291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227bb7e-f017-4c89-b16b-8128b9a41cdb None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.378426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd848439-ae81-4463-a91b-a0100ef5dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.379053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848439-ae81-4463-a91b-a0100ef5dd8d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.379336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848439-ae81-4463-a91b-a0100ef5dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.379608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848439-ae81-4463-a91b-a0100ef5dd8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.397122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6103f-db70-4946-9fa7-02ab0a877d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.397816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6103f-db70-4946-9fa7-02ab0a877d0e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.398101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6103f-db70-4946-9fa7-02ab0a877d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.398381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6103f-db70-4946-9fa7-02ab0a877d0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.462481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd848439-ae81-4463-a91b-a0100ef5dd8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.463848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd848439-ae81-4463-a91b-a0100ef5dd8d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.474531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6103f-db70-4946-9fa7-02ab0a877d0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.475960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6103f-db70-4946-9fa7-02ab0a877d0e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.481461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.515531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.559758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 215/440] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 14:44:14.560984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea9cde-fd24-4499-828f-9b206419c2d7 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.561631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea9cde-fd24-4499-828f-9b206419c2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.561631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea9cde-fd24-4499-828f-9b206419c2d7 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.562968 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 216/441] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:14.565395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea9cde-fd24-4499-828f-9b206419c2d7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.577548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 226/442] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:14.578979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5502-c87d-4a7c-aabe-32311c60a0fb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.579260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5502-c87d-4a7c-aabe-32311c60a0fb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.579527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5502-c87d-4a7c-aabe-32311c60a0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.597135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-475b5502-c87d-4a7c-aabe-32311c60a0fb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:14.598340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-475b5502-c87d-4a7c-aabe-32311c60a0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:14.662875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 227/443] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:14.663301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea9cde-fd24-4499-828f-9b206419c2d7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.663528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b5502-c87d-4a7c-aabe-32311c60a0fb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.690459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.708028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.708649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.708926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.709215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.710316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.712295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.714008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 217/444] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:14.719205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.740181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.773916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b5502-c87d-4a7c-aabe-32311c60a0fb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.809953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.811392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.811968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.812243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] PATH_INFO: `/v3/projects/9b97fbead9af485db8541ad29cc7fe55` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.813439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] RBAC: Authorizing `identity:get_project(project_id=9b97fbead9af485db8541ad29cc7fe55)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.815105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.816879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 228/445] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/projects/9b97fbead9af485db8541ad29cc7fe55 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:14.817507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43d829-5d43-45e0-9db9-28826673c7d8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.822614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.853231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.853880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.854172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.854428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.855542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.856969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.858639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 218/446] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:14.864271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.921928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:14.931393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9eaae-c442-4133-9b71-9b4f6f795309 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.965295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:14.965903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:14.966178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:14.966445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] PATH_INFO: `/v3/groups/78d06f724ade4ce4be3a0587c90bf520` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:14.967182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96199d-2873-4234-aaa1-9568b37b6e71 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:14.967604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] RBAC: Authorizing `identity:get_group(group_id=78d06f724ade4ce4be3a0587c90bf520)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:14.968958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:14.970531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 229/447] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/groups/78d06f724ade4ce4be3a0587c90bf520 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:14.971352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:14.980187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:15.004311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:15.004912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:15.005191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:15.005487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:15.006205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=9b97fbead9af485db8541ad29cc7fe55) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:15.006511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:15.007946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:15.016934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 219/448] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:15.017782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:15.084235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61945d3-2cbb-41d8-8d9c-ea4a35cebad7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:15.121922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:15.122595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:15.122832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:15.123101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:15.132734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10eb83e-1330-44db-a7ad-e82b15858394 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:15.134786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:15.140773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:15.141858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:15.142652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 230/449] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:15.148240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:15.156487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:15.177383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:15.178103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:15.178772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:15.178772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:15.179507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] RBAC: Adding query filter params (group.id=78d06f724ade4ce4be3a0587c90bf520, role.id=72e8b68a7d364c4bb0fcf9e444949d58, scope.project.id=9b97fbead9af485db8541ad29cc7fe55) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:15.179812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:15.182165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:15.188856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 220/450] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:15 2022] GET /identity/v3/role_assignments?group.id=78d06f724ade4ce4be3a0587c90bf520&scope.project.id=9b97fbead9af485db8541ad29cc7fe55&role.id=72e8b68a7d364c4bb0fcf9e444949d58 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:15.228705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:15.269407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b716e0-d071-4fa6-824c-b3d68ee78708 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:15.305947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:15.306580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:15.306842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:15.307112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:15.307781 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:15.308104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:15.309519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:15.318442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 231/451] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:15.324939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:15.427633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926c753-2c03-419a-9b7f-2359e2238cef None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:15.461297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:15.461912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:15.462188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:15.462461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:15.512676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:15.531936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:15.533415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:15.635435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 221/452] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 14:44:15.641509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:15.741287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf43a03-70dc-4027-88e3-cb3daf943dbc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:15.767223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:15.774910 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:15.775528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:15.775808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:15.776082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] PATH_INFO: `/v3/projects/5aff011fa0544761b3d08887135e99aa` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:15.777216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] RBAC: Authorizing `identity:get_project(project_id=5aff011fa0544761b3d08887135e99aa)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:15.778710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:15.780319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 232/453] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:44:15 2022] GET /identity/v3/projects/5aff011fa0544761b3d08887135e99aa => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:15.785915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:15.863262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:15.881202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3f206-aee4-441f-bdc8-1bef2fbc17a3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:15.913920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a5e9a-be4a-4712-9044-4eedaa75398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:15.914538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a5e9a-be4a-4712-9044-4eedaa75398a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:15.914816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a5e9a-be4a-4712-9044-4eedaa75398a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:15.915084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a5e9a-be4a-4712-9044-4eedaa75398a None admin] PATH_INFO: `/v3/groups/aa14a55146c041eb919c1ed610d5d5a2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:15.916169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a5e9a-be4a-4712-9044-4eedaa75398a None admin] RBAC: Authorizing `identity:get_group(group_id=aa14a55146c041eb919c1ed610d5d5a2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:15.917618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a5e9a-be4a-4712-9044-4eedaa75398a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:15.919172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 222/454] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 14:44:15 2022] GET /identity/v3/groups/aa14a55146c041eb919c1ed610d5d5a2 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:15.924973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:16.022448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3358f-c4e7-4fda-8c53-5924a94ab7d3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:16.054357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1f7c9-9104-41d5-aa2d-eeacbf87f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:16.055440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1f7c9-9104-41d5-aa2d-eeacbf87f896 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:16.055727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1f7c9-9104-41d5-aa2d-eeacbf87f896 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:16.055997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1f7c9-9104-41d5-aa2d-eeacbf87f896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:16.056712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1f7c9-9104-41d5-aa2d-eeacbf87f896 None admin] RBAC: Adding query filter params (group.id=aa14a55146c041eb919c1ed610d5d5a2, role.id=82245735988747a4bc5704727a1cf374, scope.project.id=5aff011fa0544761b3d08887135e99aa) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:16.057025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1f7c9-9104-41d5-aa2d-eeacbf87f896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:16.058502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1f7c9-9104-41d5-aa2d-eeacbf87f896 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:16.064023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 233/455] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 14:44:15 2022] GET /identity/v3/role_assignments?group.id=aa14a55146c041eb919c1ed610d5d5a2&scope.project.id=5aff011fa0544761b3d08887135e99aa&role.id=82245735988747a4bc5704727a1cf374 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:18.933020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f0cde-8fca-4129-95da-cf7d88168547 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:18.933595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f0cde-8fca-4129-95da-cf7d88168547 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:18.933595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f0cde-8fca-4129-95da-cf7d88168547 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:18.935183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 223/456] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:18.940551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c241ad7-e2e4-41a9-bace-4b95a26ec07f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:18.941434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c241ad7-e2e4-41a9-bace-4b95a26ec07f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:18.941938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c241ad7-e2e4-41a9-bace-4b95a26ec07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:18.960278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7c241ad7-e2e4-41a9-bace-4b95a26ec07f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:18.962330 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7c241ad7-e2e4-41a9-bace-4b95a26ec07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:18.976066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924536e3-1132-4498-ac97-f35ca4152c4c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:18.976349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924536e3-1132-4498-ac97-f35ca4152c4c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:18.977568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924536e3-1132-4498-ac97-f35ca4152c4c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:18.979113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 224/457] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:18.984724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b2250-c594-453e-a125-1a31bee84545 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:18.985011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b2250-c594-453e-a125-1a31bee84545 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:18.985352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b2250-c594-453e-a125-1a31bee84545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.013527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-805b2250-c594-453e-a125-1a31bee84545 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:19.014727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-805b2250-c594-453e-a125-1a31bee84545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:19.030494 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:19.045585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 234/458] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:19.106147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c7f07-94c1-433d-bb1c-f3efb7c482f5 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.106969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c7f07-94c1-433d-bb1c-f3efb7c482f5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.107477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c7f07-94c1-433d-bb1c-f3efb7c482f5 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.110037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 235/459] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:19.117853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76ff1-ad5b-4fec-9a66-72c184eb98c2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.118403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76ff1-ad5b-4fec-9a66-72c184eb98c2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.119252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76ff1-ad5b-4fec-9a66-72c184eb98c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.141459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dfc76ff1-ad5b-4fec-9a66-72c184eb98c2 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:19.143644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dfc76ff1-ad5b-4fec-9a66-72c184eb98c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:19.151774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 225/460] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:19.238595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 236/461] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:19.441940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b2250-c594-453e-a125-1a31bee84545 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:19.447565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76ff1-ad5b-4fec-9a66-72c184eb98c2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:19.560636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:19.586596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b2250-c594-453e-a125-1a31bee84545 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:19.596543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76ff1-ad5b-4fec-9a66-72c184eb98c2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:19.622685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835778e-a323-406a-b637-f4cd39986ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:19.623695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835778e-a323-406a-b637-f4cd39986ddd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.624184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835778e-a323-406a-b637-f4cd39986ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.624319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:19.624627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835778e-a323-406a-b637-f4cd39986ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.635785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835778e-a323-406a-b637-f4cd39986ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:19.637596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835778e-a323-406a-b637-f4cd39986ddd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:19.640754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:19.641408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.641689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.641958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.652303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c835778e-a323-406a-b637-f4cd39986ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:19.652597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:19.653527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:19.654380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:19.654380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:19.654380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:19.654548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:19.658702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 226/462] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:19.659959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121254ae-0ee9-4729-9d06-db734397d0cb None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.660254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121254ae-0ee9-4729-9d06-db734397d0cb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.660531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121254ae-0ee9-4729-9d06-db734397d0cb None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.661824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 227/463] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:19.662504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121254ae-0ee9-4729-9d06-db734397d0cb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:19.673952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:19.674946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:19.676813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:19.676813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:19.676813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:19.676813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 237/464] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:19.676813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:19.774276 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121254ae-0ee9-4729-9d06-db734397d0cb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:19.781094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396f1bb-ea51-427b-acb4-2d01d48ea056 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:19.806264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:19.812521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:19.813092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.813405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.813721 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.815747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8dbb6-fcfe-41cd-82d3-ea402d57c583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:19.816353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8dbb6-fcfe-41cd-82d3-ea402d57c583 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.816630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8dbb6-fcfe-41cd-82d3-ea402d57c583 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.816903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8dbb6-fcfe-41cd-82d3-ea402d57c583 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.817566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8dbb6-fcfe-41cd-82d3-ea402d57c583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:19.817870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8dbb6-fcfe-41cd-82d3-ea402d57c583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:19.819231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8dbb6-fcfe-41cd-82d3-ea402d57c583 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:19.825490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:19.826927 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:19.837243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 238/465] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:19.837243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817fa3-0586-4dbf-a9c1-537f40bc5a17 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:19.837243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817fa3-0586-4dbf-a9c1-537f40bc5a17 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:19.837243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817fa3-0586-4dbf-a9c1-537f40bc5a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:19.848447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:19.849556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:19.850569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:19.850569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:19.850569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:19.850569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 228/466] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:19.850569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:19.858351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7a817fa3-0586-4dbf-a9c1-537f40bc5a17 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:19.859558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7a817fa3-0586-4dbf-a9c1-537f40bc5a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:19.890816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:19.938300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 239/467] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:19.946361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a817fa3-0586-4dbf-a9c1-537f40bc5a17 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:19.962447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a0b8-b18e-4b58-a946-2731a97b2378 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.005496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.006556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.007017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.007794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.008950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:20.009516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.011066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.026877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 229/468] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:20.027474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:20.059539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a817fa3-0586-4dbf-a9c1-537f40bc5a17 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.080470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:20.095258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.095863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.096141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.096410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.097040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.098508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.120882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 240/469] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:44:19 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:20.121655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:20.148779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:20.160638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbefb-1923-4307-96cc-1149b20ecca0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.209317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33411305-b38b-4e53-bead-d2b0d6606437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.209933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411305-b38b-4e53-bead-d2b0d6606437 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.210363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411305-b38b-4e53-bead-d2b0d6606437 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.210825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411305-b38b-4e53-bead-d2b0d6606437 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.211987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33411305-b38b-4e53-bead-d2b0d6606437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:20.212507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33411305-b38b-4e53-bead-d2b0d6606437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.214889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33411305-b38b-4e53-bead-d2b0d6606437 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.228174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 230/470] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:20.229739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2681b-f6c4-44dd-8b7b-0855b9cf523a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.230021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2681b-f6c4-44dd-8b7b-0855b9cf523a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.230287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2681b-f6c4-44dd-8b7b-0855b9cf523a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.231909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 231/471] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:20.234363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2681b-f6c4-44dd-8b7b-0855b9cf523a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:20.258378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906fd5d-1595-4db2-af8e-55b61fc8e77c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.299271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.299271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.299532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.299626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.300265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.301745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.323276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 241/472] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:44:20 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:20.324364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:20.377868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2681b-f6c4-44dd-8b7b-0855b9cf523a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.380362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:20.423358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b41779-8e94-417a-a114-a6760534a56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.423571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41779-8e94-417a-a114-a6760534a56c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.423785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41779-8e94-417a-a114-a6760534a56c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.424067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41779-8e94-417a-a114-a6760534a56c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.436352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b41779-8e94-417a-a114-a6760534a56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.439770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b41779-8e94-417a-a114-a6760534a56c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.439927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a3b41779-8e94-417a-a114-a6760534a56c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:20.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:20.463928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:20.464858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:20.464858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:20.464858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:20.464858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 232/473] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:20.465467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712c644-6220-431d-9be9-54b379f26ee5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.465748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712c644-6220-431d-9be9-54b379f26ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.466017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712c644-6220-431d-9be9-54b379f26ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.473552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1b56d-dca9-470a-b993-36bee1a24a14 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.487444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3712c644-6220-431d-9be9-54b379f26ee5 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:20.488666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3712c644-6220-431d-9be9-54b379f26ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:20.511142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.512303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.512783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.513250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.514066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.515696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.540075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 242/474] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:44:20 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:20.540738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:20.572334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 233/475] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:20.694986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31840ae1-aacf-4bb4-a570-d09c33311d68 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.729517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:20.745112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.746316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.746596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.746867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.747543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:20.747848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.749303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.772729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 243/476] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:20.778899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712c644-6220-431d-9be9-54b379f26ee5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:20.871203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:20.881112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:20.898086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712c644-6220-431d-9be9-54b379f26ee5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:20.942640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:20.943263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:20.943565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:20.944280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:20.945292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:20.947454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:20.977217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 234/477] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:44:20 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:21.001600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942755d3-7154-4bc0-8dbc-9240137b476a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:21.044929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:21.045665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:21.046015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:21.046295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:21.047004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:21.048436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:21.060281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.sql.core [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 14:44:21.060281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 14:44:21.060281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [parameters: {'id': '8688e343b70e4d77a59ccbd89fbb975d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 14:44:21.060281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81444) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application conn.query(q) Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result.read() Sep 09 14:44:21.079791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 14:44:21.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application next(self.gen) Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application yield resource Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application next(self.gen) Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise self.value Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application session.commit() Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 14:44:21.082580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.session.flush() Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application compat.raise_( Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 14:44:21.084613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 14:44:21.086669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application conn.query(q) Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result.read() Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 14:44:21.088572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [parameters: {'id': '8688e343b70e4d77a59ccbd89fbb975d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:21.091405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 14:44:21.092362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:21.093426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 244/478] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 14:44:20 2022] POST /identity/v3/roles => generated 144 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:21.103282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:21.236936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bf575-8134-4851-8313-559ac9487f8f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:21.272351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042427df-5b5e-4b3e-a0fe-b4225f7df806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:21.273133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042427df-5b5e-4b3e-a0fe-b4225f7df806 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:21.273564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042427df-5b5e-4b3e-a0fe-b4225f7df806 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:21.273936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042427df-5b5e-4b3e-a0fe-b4225f7df806 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:21.284533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042427df-5b5e-4b3e-a0fe-b4225f7df806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-042427df-5b5e-4b3e-a0fe-b4225f7df806 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:21.290282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:21.291332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 14:44:21.292279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:21.294196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 235/479] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:44:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:21.300306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:21.369567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:21.414851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c00eb-befa-4d15-abcc-9e7a451f27ab None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:21.451523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19289f25-bcbe-47f9-8ea0-d1a44ffaee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:21.452297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289f25-bcbe-47f9-8ea0-d1a44ffaee39 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:21.452672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289f25-bcbe-47f9-8ea0-d1a44ffaee39 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:21.453041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289f25-bcbe-47f9-8ea0-d1a44ffaee39 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:21.453845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19289f25-bcbe-47f9-8ea0-d1a44ffaee39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:21.454243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19289f25-bcbe-47f9-8ea0-d1a44ffaee39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:21.455725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19289f25-bcbe-47f9-8ea0-d1a44ffaee39 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:21.465043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 245/480] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:22.725382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892c97a-2e77-4254-bfe6-f5303ee50416 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:22.725899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892c97a-2e77-4254-bfe6-f5303ee50416 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:22.725899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892c97a-2e77-4254-bfe6-f5303ee50416 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:22.727553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 236/481] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:22.732928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb680ec-ee41-4ebb-836e-8c2f3fe1b2af None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:22.733386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb680ec-ee41-4ebb-836e-8c2f3fe1b2af None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:22.733525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb680ec-ee41-4ebb-836e-8c2f3fe1b2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:22.758802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dbb680ec-ee41-4ebb-836e-8c2f3fe1b2af None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:22.760031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dbb680ec-ee41-4ebb-836e-8c2f3fe1b2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:22.839944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 246/482] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:23.115296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0f77a-5f43-4640-81c7-2859ed7bab9a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.115620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0f77a-5f43-4640-81c7-2859ed7bab9a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.115845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0f77a-5f43-4640-81c7-2859ed7bab9a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.118033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 237/483] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:23.123394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbaf10-c943-4ea5-940d-e619928ab3c1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.123686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbaf10-c943-4ea5-940d-e619928ab3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.123961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbaf10-c943-4ea5-940d-e619928ab3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.143679 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0f77a-5f43-4640-81c7-2859ed7bab9a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:23.146585 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:23.162460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-00fbaf10-c943-4ea5-940d-e619928ab3c1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:23.163703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-00fbaf10-c943-4ea5-940d-e619928ab3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:23.260822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:23.308390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 247/484] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:23.310622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3898b51-664f-4219-be0c-76f35ea4e35a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.311399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3898b51-664f-4219-be0c-76f35ea4e35a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.311867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3898b51-664f-4219-be0c-76f35ea4e35a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.318214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 248/485] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:23.319992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ff752-c183-41a1-a28b-008442e4f511 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.321054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ff752-c183-41a1-a28b-008442e4f511 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.321563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ff752-c183-41a1-a28b-008442e4f511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.341709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0f77a-5f43-4640-81c7-2859ed7bab9a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:23.342613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-131ff752-c183-41a1-a28b-008442e4f511 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:23.344255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-131ff752-c183-41a1-a28b-008442e4f511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:23.413934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:23.414607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.414886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.415401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.427725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 249/486] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:23.446686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:23.452112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:23.453110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:23.454738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 238/487] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:23.459007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ff752-c183-41a1-a28b-008442e4f511 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:23.585427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:23.610177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ff752-c183-41a1-a28b-008442e4f511 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:23.619509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:23.672885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:23.673517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.673795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.674059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.674724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:23.675021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:23.676384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:23.689964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 250/488] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:23.695452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:23.702560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:23.780817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39a94-d6cb-4553-8972-9540e3ed6274 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:23.833245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:23.834030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.834423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.834785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.848325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:23.848918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ebd7c-a720-40c0-a1e4-7e5c124fe46d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:23.854003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:23.855019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:23.855871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:23.856472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 239/489] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:23.857385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:23.890618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:23.891609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:23.892070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:23.892541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:23.905691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:23.907669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:23.925627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:23.926591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:23.927453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:23.927453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:23.927453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:23.934197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 251/490] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:23.935440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:23.959473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:23.981016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62786-1b73-4e14-a71f-1631d1c304b9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.019652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.021986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.022904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.023259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.023661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.036619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:24.043180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:24.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:24.045228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:24.046252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 240/491] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:24.047083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.061460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601475f5-8460-4f21-a4bf-e31fefb23a58 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.101792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be24340-be1d-4fb9-8f63-95eb9978e089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.102609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be24340-be1d-4fb9-8f63-95eb9978e089 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.103030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be24340-be1d-4fb9-8f63-95eb9978e089 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.103503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be24340-be1d-4fb9-8f63-95eb9978e089 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.104431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be24340-be1d-4fb9-8f63-95eb9978e089 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:24.104881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be24340-be1d-4fb9-8f63-95eb9978e089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.106642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be24340-be1d-4fb9-8f63-95eb9978e089 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.115375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 252/492] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:24.116995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab57e29-ca84-4896-b89b-03a53807bff7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.117856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab57e29-ca84-4896-b89b-03a53807bff7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.118136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab57e29-ca84-4896-b89b-03a53807bff7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.119490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 253/493] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:24.120184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab57e29-ca84-4896-b89b-03a53807bff7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.155859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a012c6-29df-435f-a89b-7211b2c1c3e4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.191171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.191783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.192047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.192317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.192948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:24.193262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.194636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.207259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 241/494] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:24.207774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.208051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.232491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab57e29-ca84-4896-b89b-03a53807bff7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.255254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.268420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716597a0-1e25-4105-9178-f07d83fd24ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.269035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716597a0-1e25-4105-9178-f07d83fd24ea None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.269346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716597a0-1e25-4105-9178-f07d83fd24ea None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.269621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716597a0-1e25-4105-9178-f07d83fd24ea None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.270280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716597a0-1e25-4105-9178-f07d83fd24ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:24.270579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716597a0-1e25-4105-9178-f07d83fd24ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.271961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716597a0-1e25-4105-9178-f07d83fd24ea None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.280373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 254/495] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:24.281431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b25b9-ecfa-4aac-865b-99713038fb7f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.281714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b25b9-ecfa-4aac-865b-99713038fb7f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.281985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b25b9-ecfa-4aac-865b-99713038fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.298989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6e3b25b9-ecfa-4aac-865b-99713038fb7f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:24.300163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6e3b25b9-ecfa-4aac-865b-99713038fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:24.318861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bece901-1134-4c80-b955-757a7fb5645d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.359590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.360148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.360422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.360684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.371879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.373359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.376116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 255/496] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:24.377047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b25b9-ecfa-4aac-865b-99713038fb7f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:24.403522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:24.404492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:24.405355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:24.405355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:24.405355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:24.406708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 242/497] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:24.407505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.460360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.489630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b25b9-ecfa-4aac-865b-99713038fb7f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.514052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.521494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa893c8-4bdb-4a0a-9ce0-56a19850b73b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.526308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.526941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.527222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.527500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.535422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.536872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:24.554779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:24.556229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 14:44:24.557132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:24.558294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641d44b-23f2-407e-bb44-1d7b26caa816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.558294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641d44b-23f2-407e-bb44-1d7b26caa816 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.558294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641d44b-23f2-407e-bb44-1d7b26caa816 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.558294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641d44b-23f2-407e-bb44-1d7b26caa816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.558836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 256/498] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:24.558836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.575957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641d44b-23f2-407e-bb44-1d7b26caa816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.578401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641d44b-23f2-407e-bb44-1d7b26caa816 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f641d44b-23f2-407e-bb44-1d7b26caa816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:24.593436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:24.594359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:24.595199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:24.595199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:24.595199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:24.597465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 243/499] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:24.598510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442dde1-9b9b-4949-849d-81d39125014c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.598791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442dde1-9b9b-4949-849d-81d39125014c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.599062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442dde1-9b9b-4949-849d-81d39125014c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.600303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 244/500] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:24.601009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442dde1-9b9b-4949-849d-81d39125014c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.669062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6b2e5-7c4f-44bf-8c8a-a441afd36ad9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.701813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.702402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.702679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.702942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.703573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:24.703868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.705241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.705341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442dde1-9b9b-4949-849d-81d39125014c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.706782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.718381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 257/501] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:24.720309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.738757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f9d2f-9021-4d0e-badf-9df885f4fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.739361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9d2f-9021-4d0e-badf-9df885f4fcdf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.739637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9d2f-9021-4d0e-badf-9df885f4fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.739910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9d2f-9021-4d0e-badf-9df885f4fcdf None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.740627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f9d2f-9021-4d0e-badf-9df885f4fcdf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:24.740929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f9d2f-9021-4d0e-badf-9df885f4fcdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.742995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f9d2f-9021-4d0e-badf-9df885f4fcdf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.778943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 245/502] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:24.780201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633438f7-558e-416a-8ca5-032b551b36fe None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.780563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633438f7-558e-416a-8ca5-032b551b36fe None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.780751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633438f7-558e-416a-8ca5-032b551b36fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.787650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.799699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-633438f7-558e-416a-8ca5-032b551b36fe None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:24.800887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-633438f7-558e-416a-8ca5-032b551b36fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:24.818073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf5867-fe68-49ca-a2dc-1aa16f3777c0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.852337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:24.852961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:24.853232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:24.853505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:24.854143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:24.854444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:24.855808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:24.866543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 246/503] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:24.866772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 258/504] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:24.867582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.867703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633438f7-558e-416a-8ca5-032b551b36fe None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:24.929352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:24.972098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633438f7-558e-416a-8ca5-032b551b36fe None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:24.973463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b0018-d8af-4074-84ca-27fa717c3dc6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.006305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.006926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.007205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.007478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.012086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.012086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.012086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.012086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.015161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.021953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.024054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.027781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:25.032559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:25.033457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:25.034282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 247/505] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:25.034952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:25.040241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:25.041148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:25.042081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:25.045322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 259/506] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:44:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:25.045322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.144951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68999580-e3e0-461d-bd4e-2f72c5ec3a6b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.145471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474e1f2-1dbd-4adb-b7f6-0c4832ade8be None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.159276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.180047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.180581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.180861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.181135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.181411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.182034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.182293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.182560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.189792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.191242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:25.202583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:25.203447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:25.204323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 260/507] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:25.204920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.241584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.249164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.250652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.306324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5ee91-ee0c-4ae7-8947-216a752ad1d4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.339779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.340379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.340657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.340926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.341628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:25.341914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.343282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.348353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 248/508] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:25.349174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.351394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 261/509] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:25.352105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.390048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.441086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.448570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b258ac0-c0c9-4847-bf9d-74740cb8a926 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.453744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f4a24-d68b-4ebe-a5ce-bddd6f77c00a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.481986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.482590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.482869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.483178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.483847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:25.484155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.485606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.486096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.486721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.487089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.487261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.497848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.499264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.502689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 249/510] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:25.503475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:25.514102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:25.515115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:25.515929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:25.515929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:25.515929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:25.515929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 262/511] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:25.515929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.601707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528a160-7867-4ce4-bc65-857e141b23d7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.602932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.614830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158c6d-dce0-475d-b2cb-e0b22856a0a1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.635283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.635882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.636156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.636680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.637428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:25.637729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.639159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.648938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.649582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.649863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.650136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.656086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 250/512] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:25.656877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.665067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.666917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.668015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:25.682109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:25.683193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:25.684241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:25.684241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:25.684241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:25.684241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 263/513] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:25.684241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.756631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9555d33-89a4-4d0b-b8da-bdb5787fbbd1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.778890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61802a38-f721-4cbb-bf7c-638583f4f0bc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.790942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.791595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.791878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.792151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.802048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.803477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.812196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.812790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.813070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.813376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:25.817884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:25.818906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:25.819739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:25.819739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:25.819739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:25.819739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 251/514] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:25.820117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.824118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.883302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.884069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:25.884803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.923868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b8643-2d0c-4277-9ec2-9040534d3f01 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:25.957742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:25.958340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:25.958620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:25.958895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:25.959537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:25.959840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:25.961874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:25.973291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 252/515] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:25.974077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:25.983119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 264/516] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:25.984215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.028232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.078355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c17bf3-3ab5-4efd-8b2e-13d6829f0aab None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.084736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632de72-7d70-4fae-8ee0-9263653c1719 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.102614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.114743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.115346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.115625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.115900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.118520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.119109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.119392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.119657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.120286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:26.120582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.122036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.132644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 265/517] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:26.134032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.182588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.184030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.229141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.232219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc7ed2-665a-4b80-97f1-ff5a8e9d78a6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.265833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.266369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.266646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.266922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.267549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:26.267854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.269242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.280821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 266/518] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:26.281603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.286652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 253/519] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:26.286652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.306591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.379934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b6da9-e250-4efb-842b-fb39683268f4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.383855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9639bfa-e6fe-41d9-b8f0-06b4def5eb4b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.412697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.413336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.413616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.413879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.416344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.416935 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.417236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.417514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.418147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.419573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.423755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.425181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:26.440070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:26.441035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:26.442403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:26.442403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:26.442403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:26.442403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 254/520] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:26.442403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.447786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 267/521] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:44:26 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:26.448722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.469091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.535950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.565056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b232b-0053-4f13-88b5-35d2f371b3cf None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.566583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef9d37-b086-4a0c-840b-5d3e4e6baf8c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.602601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.603158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.603689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.603805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.604061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.604332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.604530 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.604940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.611923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.613351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.613505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.614810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:26.626305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:26.627133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:26.632288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:26.632945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:26.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 14:44:26.635157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:26.635652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 255/522] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:26.635718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 268/523] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:26.635781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.635853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.718051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.745153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9736c-7ecc-4748-abf1-bf6100f5860b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.748842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72e88c-83f2-48aa-9b08-ffe9fbbc8bbb None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.779719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.780410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.780690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.780964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.782906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.786853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.787786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.788270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.788546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.789867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:26.790170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.791815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.792832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.794370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.804324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 269/524] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:26.805949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:26.810105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:26.811009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:26.811844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:26.811844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:26.811844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:26.811844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 256/525] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:26.812075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.911956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bcfaa-fc12-4431-add2-3517448f2762 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.919757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406b57c-6d41-40f9-85bd-b9e7901dc464 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:26.948022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.948626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.948903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.949177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.949883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:26.950190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.951639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.955206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:26.955818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:26.955947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:26.956086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:26.956755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:26.957582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:26.957925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:26.959377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:26.975476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 270/526] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:26.976144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:26.984248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 257/527] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:26.984248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.037705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:27.083532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402c1f-b3c5-4d0a-a28a-9c432e95479d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.095491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779a34-c34c-4966-85c7-e200b0f80963 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.119812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.120437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.120717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.120992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.127662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.129117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.134104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.135158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.135538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.135901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.136935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:27.137353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.138913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:27.141188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:27.142055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:27.142881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 271/528] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:27.143335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.155322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 258/529] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:27.156662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.201255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:27.254383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc13181-972a-49a9-9ec5-389684beb7d6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.274817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab798bc-87ec-44f9-b153-281fa2406fe4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.277124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:27.293286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.293984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.294258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.294525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.295248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=04d1fc63b356499d80061e358fb447e4) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:27.295552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.297099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.313233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 272/530] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/role_assignments?user.id=04d1fc63b356499d80061e358fb447e4&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:27.313233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.323104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.323912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.324293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.324666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.406819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.408463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.459981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f56424-77e5-4715-a0ff-14ee9ffd9a53 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.490499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:27.507599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.508736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.509015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.510904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.510904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:27.510904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.512139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.526085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 273/531] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:27.526929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.540566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 259/532] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:27.542277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.546269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:27.657536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206a667-b1db-4001-bf2e-a78e4e412d58 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.672736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f5315-7c21-446b-9398-e60d5429290e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.700565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.702144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.702734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.703314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.709651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.710345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.710688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.711584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.712430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=7b6809bf91d244e4bed5b8a256a3f95a) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:27.712726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.713475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.714321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.715331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.720509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 260/533] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/role_assignments?user.id=7b6809bf91d244e4bed5b8a256a3f95a&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:27.721530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:27.731375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:27.732305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:27.733139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:27.736906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 274/534] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:27.742738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:27.808350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:27.853678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113a646-8b03-4c78-9b8c-e12b82dfbe7b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:27.888739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:27.906170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:27.907284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:27.907668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:27.907947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:27.936563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:27.938232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:27.955867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:27.956983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:27.958050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:27.958050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:27.958050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:27.958050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 261/535] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:27.963923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:28.001129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf0a2a-0773-499a-9931-5cb99168bb48 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:28.087528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab594-4d98-490d-922b-598dafe155ba None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:28.098035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:28.098663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.098944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.099216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.099915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:28.100221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:28.101781 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:28.109706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:28.123514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 275/536] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:28.131719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:28.132388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.132728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.134013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.134730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:28.134839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:28.135031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:28.136835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:28.151655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 262/537] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:28.157507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:28.273891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924be7c1-c222-4329-b7f4-ae0f604fc9ce None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:28.314237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:28.315003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.315286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.315557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.323130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:28.325114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:28.337663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:28.338485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:28.339318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 263/538] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:28.344500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:28.370544 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:28.373619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b80c-5a09-4f72-a85a-90a972cc7372 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:28.405502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:28.446546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:28.447218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.447505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.447776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.448565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=b46482f26051455daeed2c00f8ced4bd) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:28.448810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:28.450319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:28.467659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 276/539] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:28 2022] GET /identity/v3/role_assignments?user.id=b46482f26051455daeed2c00f8ced4bd&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:28.474845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dd539-3c67-4e07-a16b-d660c8e4cf3d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:28.541117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bfde1-67d6-4d58-9f08-64b4bc1aee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:28.541920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bfde1-67d6-4d58-9f08-64b4bc1aee2c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.542290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bfde1-67d6-4d58-9f08-64b4bc1aee2c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.542672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bfde1-67d6-4d58-9f08-64b4bc1aee2c None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.550766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bfde1-67d6-4d58-9f08-64b4bc1aee2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:28.551336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bfde1-67d6-4d58-9f08-64b4bc1aee2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:28.552937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bfde1-67d6-4d58-9f08-64b4bc1aee2c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:28.577920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 264/540] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:28.583793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:28.736822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2c2f4-4cd6-42a2-b656-09d6dbf432f1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.737341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2c2f4-4cd6-42a2-b656-09d6dbf432f1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.737729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2c2f4-4cd6-42a2-b656-09d6dbf432f1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.740199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 265/541] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:28.747487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37443d-dad2-4651-abff-674f4c56affb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.747882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37443d-dad2-4651-abff-674f4c56affb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.748253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37443d-dad2-4651-abff-674f4c56affb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.750839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72914954-6866-4be5-a8b1-f211f5e0d65a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:28.790412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:28.791259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:28.791675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6e37443d-dad2-4651-abff-674f4c56affb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:28.791853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:28.792316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:28.793184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6e37443d-dad2-4651-abff-674f4c56affb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:28.793314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=f9edd8cf448e42fd8e888c8f531fa70d) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:28.793742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:28.795640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:28.802027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 277/542] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:28 2022] GET /identity/v3/role_assignments?user.id=f9edd8cf448e42fd8e888c8f531fa70d&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:28.899393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 266/543] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:29.220914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:29.257744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:29.351159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f202f8-4aa5-48ff-b9dc-ce80789b9478 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:29.400788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:29.401444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:29.401721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:29.401990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:29.422124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:29.427386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:29.428250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:29.429029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 278/544] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:29.435191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37443d-dad2-4651-abff-674f4c56affb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:29.488622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:29.584347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37443d-dad2-4651-abff-674f4c56affb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:29.634560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16883f95-f2b7-4074-9ff7-c216280501d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:29.635170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16883f95-f2b7-4074-9ff7-c216280501d8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:29.635445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16883f95-f2b7-4074-9ff7-c216280501d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:29.635713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16883f95-f2b7-4074-9ff7-c216280501d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:29.636639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16883f95-f2b7-4074-9ff7-c216280501d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:29.636937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16883f95-f2b7-4074-9ff7-c216280501d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:29.638339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16883f95-f2b7-4074-9ff7-c216280501d8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:29.646841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 267/545] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:29.654276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:29.793637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:29.811986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c3cd0-f56a-462c-9e6e-dff7d0a2c210 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:29.854875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb2936-9919-4b38-a27b-5d1bb2d4924a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:29.855363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb2936-9919-4b38-a27b-5d1bb2d4924a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:29.855852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb2936-9919-4b38-a27b-5d1bb2d4924a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:29.855951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:29.856568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:29.856843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:29.857111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:29.858060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 268/546] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:29.866588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d01a6-7dc4-49a6-a7a0-d16aadd571f3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:29.866588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d01a6-7dc4-49a6-a7a0-d16aadd571f3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:29.866588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d01a6-7dc4-49a6-a7a0-d16aadd571f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:29.874372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:29.875857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:29.894464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a92d01a6-7dc4-49a6-a7a0-d16aadd571f3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:29.895598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a92d01a6-7dc4-49a6-a7a0-d16aadd571f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:29.898029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:29.899007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:29.899834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:29.899834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:29.899834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:29.899834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 279/547] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:29.905231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:30.010193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 269/548] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:30.108830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf315774-0d8a-4945-b810-a117565ef76e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:30.153422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:30.153958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.154240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.154511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.155148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:30.155449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:30.156828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:30.166304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:30.170503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 280/549] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:30.176645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d01a6-7dc4-49a6-a7a0-d16aadd571f3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:30.218732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:30.310443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:30.358257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d01a6-7dc4-49a6-a7a0-d16aadd571f3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:30.401682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fc7cb-bb17-4442-aa20-2796391b446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:30.402315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fc7cb-bb17-4442-aa20-2796391b446c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.402587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fc7cb-bb17-4442-aa20-2796391b446c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.402853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fc7cb-bb17-4442-aa20-2796391b446c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.413471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fc7cb-bb17-4442-aa20-2796391b446c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:30.415208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fc7cb-bb17-4442-aa20-2796391b446c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7b5fc7cb-bb17-4442-aa20-2796391b446c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:30.431628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:30.433129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:30.434152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:30.436310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 270/550] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:30.436310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cec1c2-c8b0-43f5-8045-429c63fe69d4 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.436310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cec1c2-c8b0-43f5-8045-429c63fe69d4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.436616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cec1c2-c8b0-43f5-8045-429c63fe69d4 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.441436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 271/551] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:30.441436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cec1c2-c8b0-43f5-8045-429c63fe69d4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:30.454371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457b27-093a-43aa-8ab1-749a6b28139e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:30.495876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54a722-a581-48a9-877d-c81f08cf0697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:30.496503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a722-a581-48a9-877d-c81f08cf0697 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.496778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a722-a581-48a9-877d-c81f08cf0697 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.497047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a722-a581-48a9-877d-c81f08cf0697 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.510023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a722-a581-48a9-877d-c81f08cf0697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2f54a722-a581-48a9-877d-c81f08cf0697 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:30.516536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:30.517672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:30.519150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:30.521265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 281/552] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:30.522768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b54e-de8b-436a-99f9-67a71df768d2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.523051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b54e-de8b-436a-99f9-67a71df768d2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.524706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b54e-de8b-436a-99f9-67a71df768d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.546915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-da34b54e-de8b-436a-99f9-67a71df768d2 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:30.548293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-da34b54e-de8b-436a-99f9-67a71df768d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:30.554107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:30.576951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cec1c2-c8b0-43f5-8045-429c63fe69d4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:30.613351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:30.615675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:30.616542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.617046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.617527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.618929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:30.619699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:30.621605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:30.633359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 282/553] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:30.634451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b54e-de8b-436a-99f9-67a71df768d2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:30.641057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 272/554] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 14:44:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:30.647525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:30.756552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6d022-6bd8-419b-a34e-231e4dc8c8ad None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:30.796952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:30.798014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.798841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.799422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.822902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:30.859172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b54e-de8b-436a-99f9-67a71df768d2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:30.872577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:30.874416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:30.977929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:30.978560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:30.978841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:30.979111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:30.979792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:30.980093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:30.981084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 273/555] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:30.985304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:30.986955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.007436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 283/556] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:31.008811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.038277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.079891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.105331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0136d86-cbf3-43d5-badb-263d3af92917 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.133530 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf77d9-8dcc-4cfa-9933-d446295c8de8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.143858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.144702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.145149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.145695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.157110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.158970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:31.176466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:31.177474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:31.178411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:31.178411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:31.178411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:31.178947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.180412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.180985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.181452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.183177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 274/557] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:31.184343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.198751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:31.207080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:31.208528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:31.210273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:31.218005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 284/558] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:31.218005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.306311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.315795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dd4ee-d12d-43b0-a58f-6553be5f39a2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.348665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991d0bb-716d-460b-8cbf-6ef65b1601b6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.354145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.354774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.355047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.355312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.361581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.367298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.368740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:31.383385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:31.384481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:31.385369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:31.385369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:31.385369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:31.385369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 275/559] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:31.385369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.390764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5081d7f-96d8-4b1d-b229-fc3dd1e93157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.391365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081d7f-96d8-4b1d-b229-fc3dd1e93157 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.391641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081d7f-96d8-4b1d-b229-fc3dd1e93157 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.391908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081d7f-96d8-4b1d-b229-fc3dd1e93157 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.392546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5081d7f-96d8-4b1d-b229-fc3dd1e93157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:31.392846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5081d7f-96d8-4b1d-b229-fc3dd1e93157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.394246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5081d7f-96d8-4b1d-b229-fc3dd1e93157 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.406575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 285/560] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:31.407586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07356b-8339-4b91-970f-a41620cdb885 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.407867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07356b-8339-4b91-970f-a41620cdb885 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.408137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07356b-8339-4b91-970f-a41620cdb885 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.409465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 286/561] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:31.410333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81fd8f-1f6d-4497-bc22-4e5e6e5d7e4b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.410612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81fd8f-1f6d-4497-bc22-4e5e6e5d7e4b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.411075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81fd8f-1f6d-4497-bc22-4e5e6e5d7e4b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.412042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 287/562] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:31.412720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81fd8f-1f6d-4497-bc22-4e5e6e5d7e4b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.489794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf039d-dccf-4786-883d-78e23ea5ebe4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.511838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81fd8f-1f6d-4497-bc22-4e5e6e5d7e4b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.523197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.523818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.524152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.524421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.525109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:31.525438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.526816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.535074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 276/563] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:31.535840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.544295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.549541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7431f-03b4-415e-bad7-a4af0ee855fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.550150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7431f-03b4-415e-bad7-a4af0ee855fd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.550437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7431f-03b4-415e-bad7-a4af0ee855fd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.550708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7431f-03b4-415e-bad7-a4af0ee855fd None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.551342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7431f-03b4-415e-bad7-a4af0ee855fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:31.551642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7431f-03b4-415e-bad7-a4af0ee855fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.553006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7431f-03b4-415e-bad7-a4af0ee855fd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.564544 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 288/564] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:31.565661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d60ca4-9523-4203-8e96-b594d8b90ec2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.566016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d60ca4-9523-4203-8e96-b594d8b90ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.566209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d60ca4-9523-4203-8e96-b594d8b90ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.583484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-41d60ca4-9523-4203-8e96-b594d8b90ec2 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:31.584675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-41d60ca4-9523-4203-8e96-b594d8b90ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:31.609949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.635334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af383ff1-7207-4d0a-b41c-faf2e4824f5f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.651242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 289/565] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:31.652445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad376758-b605-472c-9098-3f1be4449f33 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.652720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad376758-b605-472c-9098-3f1be4449f33 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.652990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad376758-b605-472c-9098-3f1be4449f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.670146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ad376758-b605-472c-9098-3f1be4449f33 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:31.671312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ad376758-b605-472c-9098-3f1be4449f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:31.680519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.681146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.681513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.681767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.688982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.690478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:31.702929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:31.703759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:31.704612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 277/566] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:31.705083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.748367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 290/567] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:31.754118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad376758-b605-472c-9098-3f1be4449f33 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.779520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.814627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e9473-ee4a-48f3-8b7d-2332e2d7f265 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.848712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.849389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.849666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.849938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.851503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:31.860725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.861082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad376758-b605-472c-9098-3f1be4449f33 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:31.862298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:31.877506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:31.878440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:31.879304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:31.879304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:31.879304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:31.879304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 278/568] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:31.880039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:31.902953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:31.904011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:31.904505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:31.905160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:31.914794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:31.917334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:31.937571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:31.938575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 291/569] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:31.939499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.003951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672f4da-ac14-488c-92f6-fe96ef87faa0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.024885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.042989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.043615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.043887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.044152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.044832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:32.045129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.046580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.050271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b742f33-b294-459f-8903-7dad6267e496 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.058783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 279/570] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:32.059558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.086145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.086960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.087234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.087502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.088132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:32.088434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.089894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.100902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 292/571] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:32.102217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.116525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.159706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3f270-7702-4923-9908-d8c24e20ed30 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.193245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.193869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.194144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.194417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.198993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3063-fa8f-42d1-ab4e-db46cd0ec399 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.204093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:32.208847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:32.209799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:32.210640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 280/572] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:32.211452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.235425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.236109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.236446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.236719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.237461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:32.237773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.239175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.249837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.276628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 293/573] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:32.277523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.316292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73c5a7-a8c8-401f-9d14-aed1292df97c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.347712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.349903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.350556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.350857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.351118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.361724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:32.366548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:32.367489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:32.368308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 281/574] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:32.368978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.384835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457a1fa-c17d-4014-8767-204190e1c23c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.418728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.419323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.419598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.419867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.420573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=552b3ea5ceab4114908b8108d415c9cd) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:32.420871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.422337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.427456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 294/575] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/role_assignments?user.id=552b3ea5ceab4114908b8108d415c9cd&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:32.428216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.473621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.479261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4834c0-53d2-4435-b9fb-4dd4876dcd5c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.517779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.518472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.518758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.519035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.528252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.529788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:32.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:32.543816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 282/576] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:32.544742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.559864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939fe5-30f1-458c-9f3d-4b5124c6a070 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.598124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.598669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.598949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.599706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.599876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:32.600177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.602233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.603838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.614278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 295/577] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:32.615109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.663618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d68b56-8798-4890-966d-f1cb0e1a4b85 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.700615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.701441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.701812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.702548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.714878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.716750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.729596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71d3ec-eb5a-4a31-b736-9407bf306822 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:32.731584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:32.732481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:32.734506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:32.734506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:32.734506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:32.734506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 283/578] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:32.734506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.746197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.770750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.771491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.771884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.772219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.772939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:32.773962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.775343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.783455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 296/579] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:32.784241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.844858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544eb52-fd90-49e3-843d-f807cdabadbe None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.846056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:32.880071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.880664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.880932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.881222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.882007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:32.882311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.884257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.891543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39c539-8dda-4bfc-9776-82ced875c329 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:32.904949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 284/580] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:32.910221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.925174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:32.925886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:32.926228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:32.926563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:32.937066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:32.938551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:32.953246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:32.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:32.955269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:32.955269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:32.955269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:32.959319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 297/581] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:32.960123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:32.986431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.032162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c047cd-9795-4e04-a427-c3124116cd93 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.072991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.073750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.074003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.074271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.074901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:33.075197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.076544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.085986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b027-a7ff-45c6-8a33-bf7e1bda4a2d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.088692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 285/582] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:33.089505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.100868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.124907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.125647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.125994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.126276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.138214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.139687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:33.154255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:33.155372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:33.156283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:33.156283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:33.156283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:33.156283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 298/583] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:33.156511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.202394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebe722-f0a4-4a4f-9bdc-7dfe7c441b79 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.228613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.238582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.239288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.239639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.239978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.251567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.253077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.255938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7fd7b-f755-4213-ba7a-4f6923348544 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:33.267985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:33.269071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:33.270304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:33.270304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:33.270304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:33.270304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 286/584] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:33.271042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.294514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.295114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.295392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.295664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.296297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:33.296592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.298033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.315684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 299/585] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:33.315684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.342661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.382076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effbb7f-9d73-4aa2-8f04-8dfbcdd0123c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.419355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.419902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.420174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.420441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.421147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91c2d0-7bcd-405c-a4ff-97962bcddd52 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.426878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.428293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:33.440289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:33.441358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 287/586] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:33.442211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.461904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.462828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.463252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.463709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.464500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:33.464943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.466954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.479398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 300/587] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:33.480376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.489926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.549335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f93d99-3834-4665-9139-33cbeff9413d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.566411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.583156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.584000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.584273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.584541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.585163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:33.585491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.585779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a168bd-5f61-4b4d-97a1-00025f1d2dd4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.586870 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.605254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 288/588] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:33.605254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.626649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.627276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.627551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.627833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.635786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.637512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:33.651456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:33.652409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 301/589] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:33.653455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.714665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.718372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600f5be-4ba0-4845-86c5-85c2b0b2a449 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.754782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.755382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.755914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.755914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.756588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:33.756890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.758374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.765101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b6b64-bcc3-45ba-bf27-597de42428e2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.770523 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 289/590] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:33.771450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.803741 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.804383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.804708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.804981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.812235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.813702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:33.824836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:33.825913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 302/591] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:33.826990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.829816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.885160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8783287-c3b4-4673-9f6f-3a068892d15a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.925400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.926019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.926295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.926566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.933554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627306c-58ce-424b-ba73-ae09f9f33dcc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:33.937008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.939230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:33.952238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:33.953430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:33.954487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:33.958244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 290/592] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:33.958771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:33.971825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:33.972395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:33.972744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:33.973080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:33.973888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:33.974256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:33.975854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:33.983979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:33.999464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 303/593] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:33.999464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.066298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:34.080516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263b00c-c2a2-4621-99f1-989f1b85b328 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.113171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfe82f-4dd1-44de-9afa-6d8bc4331040 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.115066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.116550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.116902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.117291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/04d1fc63b356499d80061e358fb447e4/roles/3a9af0f0d1d04401a0fee6860911b19f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.132120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=04d1fc63b356499d80061e358fb447e4, role_id=3a9af0f0d1d04401a0fee6860911b19f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.134122 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.149667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 291/594] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:33 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/04d1fc63b356499d80061e358fb447e4/roles/3a9af0f0d1d04401a0fee6860911b19f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:34.150627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.164385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.165165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.165586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.165996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.166850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:34.167257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.168974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.190036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 304/595] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 9 14:44:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:34.194247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.245287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:34.284019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81272-befe-4edb-a861-51a3a7096927 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.324522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.325366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.325742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.326106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.326923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:34.327376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.328929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.339288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34002760-dd90-456e-8342-ae8f0d7a2e01 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.352499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 292/596] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:34.353888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.366261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:34.393115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.393810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.394090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.394360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.405635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.407098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:34.422186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:34.423853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:34.424717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:34.424717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:34.424717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:34.424717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 305/597] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:34.424717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.471062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a882f6e-8800-43d5-b728-02c7e01a4a30 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.502269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:34.509616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.510386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.510767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.511172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.522907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.524515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:34.540364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:34.541346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:34.542374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:34.542374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:34.542374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:34.542374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 293/598] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:34.543657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.551373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db621-3468-4b8d-9201-bb75f5b180ff None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.602367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.603064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.603370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.604214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/7b6809bf91d244e4bed5b8a256a3f95a/roles/3a9af0f0d1d04401a0fee6860911b19f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.627342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=7b6809bf91d244e4bed5b8a256a3f95a, role_id=3a9af0f0d1d04401a0fee6860911b19f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.628892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.648244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 306/599] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:34 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/7b6809bf91d244e4bed5b8a256a3f95a/roles/3a9af0f0d1d04401a0fee6860911b19f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:34.649580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.659387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5a677-abda-49c5-ad6e-3cdc43d676eb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.697051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:34.726728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479439cc-80bb-4c1e-9f97-746fe0583d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.727510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479439cc-80bb-4c1e-9f97-746fe0583d8c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.727935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479439cc-80bb-4c1e-9f97-746fe0583d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.728303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479439cc-80bb-4c1e-9f97-746fe0583d8c None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.729072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479439cc-80bb-4c1e-9f97-746fe0583d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:34.729578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479439cc-80bb-4c1e-9f97-746fe0583d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.731153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479439cc-80bb-4c1e-9f97-746fe0583d8c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.745672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 294/600] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:34.747159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a51b2-9785-47a4-a769-6ab081e88470 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.752364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a51b2-9785-47a4-a769-6ab081e88470 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.752682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a51b2-9785-47a4-a769-6ab081e88470 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.759274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 295/601] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:34.759274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a51b2-9785-47a4-a769-6ab081e88470 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.805716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3ef30-206b-4296-9054-2faac3a445c8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.811074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:34.844685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a7577-12bd-4002-9887-f103890bd3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.845583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7577-12bd-4002-9887-f103890bd3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.846005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7577-12bd-4002-9887-f103890bd3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.846416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7577-12bd-4002-9887-f103890bd3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.847225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a7577-12bd-4002-9887-f103890bd3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:34.847668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a7577-12bd-4002-9887-f103890bd3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.849193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a7577-12bd-4002-9887-f103890bd3e6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.862228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 307/602] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:34.863835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2106c-76c7-4b76-a74d-1d32b50bb3c2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.864278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2106c-76c7-4b76-a74d-1d32b50bb3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.864705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2106c-76c7-4b76-a74d-1d32b50bb3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.882396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a51b2-9785-47a4-a769-6ab081e88470 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:34.885358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d5a2106c-76c7-4b76-a74d-1d32b50bb3c2 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:34.886948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d5a2106c-76c7-4b76-a74d-1d32b50bb3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:34.923926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:34.924561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:34.924834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:34.925099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:34.932811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:34.934277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:34.946676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:34.947595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 296/603] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:34.948821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.966875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 308/604] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:34.967933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2106c-76c7-4b76-a74d-1d32b50bb3c2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:34.980567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:35.072007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bebf31-96d1-4c79-8ec4-f7b2b0080c9f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:35.074013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:35.096794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2106c-76c7-4b76-a74d-1d32b50bb3c2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:35.156650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:35.157302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:35.157577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:35.157847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:35.160908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:35.161532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:35.161805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:35.162076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:35.162755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:35.163052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:35.165002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:35.172489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:35.173941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:35.189756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 309/605] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:35.195626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:35.196832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:35.197740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:35.199344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:35.200442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 297/606] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:35.206186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:35.318566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:35.358526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b0dd2-0f46-4bac-bf75-ff0808e7d57b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:35.366259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e2308-1587-4bd4-884c-a964142460e7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:35.403573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:35.404480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:35.404856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:35.405345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/b46482f26051455daeed2c00f8ced4bd/roles/3a9af0f0d1d04401a0fee6860911b19f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:35.407364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:35.408283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:35.408808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:35.409282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:35.409974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:35.410284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:35.411683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:35.422860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=b46482f26051455daeed2c00f8ced4bd, role_id=3a9af0f0d1d04401a0fee6860911b19f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:35.424306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:35.428557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 298/607] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:35.429758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:35.445181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 310/608] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:35 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/b46482f26051455daeed2c00f8ced4bd/roles/3a9af0f0d1d04401a0fee6860911b19f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:44:35.454008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:35.492869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:35.659611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665a07-dac3-4afa-8298-d5320e9639a5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:35.671698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d852ad9-8603-4c4e-8422-4ac445683d29 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:35.694597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:35.702459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:35.703101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:35.703377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:35.703643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:35.713627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:35.714707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:35.715293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:35.715820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/f9edd8cf448e42fd8e888c8f531fa70d/roles/3a9af0f0d1d04401a0fee6860911b19f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:35.717160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:35.724079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:35.725089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:35.725979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 299/609] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:35.732904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:35.740257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=f9edd8cf448e42fd8e888c8f531fa70d, role_id=3a9af0f0d1d04401a0fee6860911b19f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:35.742286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:35.761632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 311/610] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:35 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/f9edd8cf448e42fd8e888c8f531fa70d/roles/3a9af0f0d1d04401a0fee6860911b19f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:35.872046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:35.899932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b707e9-c41e-4def-84d4-9d2398ca9cef None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:35.935104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:35.935625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:35.935956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:35.936225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:35.936875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:35.937170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:35.938544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:35.948077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 300/611] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:35.954212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:36.105959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6839cc-2bb1-48d8-82e2-f55c63eaa26f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:36.143015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:36.143655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:36.143937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:36.144209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:36.157920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:36.159387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:36.178739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:36.180312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:36.181287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:36.181287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:36.181287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:36.181287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 312/612] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:36.187015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:36.322000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c793e-ff4f-42a2-87e0-007dea814c22 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:36.365901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369e3e6-93b8-46bc-8e46-3b9b46118df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:36.366509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e3e6-93b8-46bc-8e46-3b9b46118df9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:36.366788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e3e6-93b8-46bc-8e46-3b9b46118df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:36.367088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e3e6-93b8-46bc-8e46-3b9b46118df9 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:36.367723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369e3e6-93b8-46bc-8e46-3b9b46118df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:36.368021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369e3e6-93b8-46bc-8e46-3b9b46118df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:36.369395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369e3e6-93b8-46bc-8e46-3b9b46118df9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:36.386039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 301/613] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:36.391803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:36.437283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:36.578008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a2eb-b9e4-4c19-8f2d-e3559779301c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:36.596241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b37791-5ab0-4b6e-9648-d80a108dae62 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:36.596534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b37791-5ab0-4b6e-9648-d80a108dae62 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:36.596780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b37791-5ab0-4b6e-9648-d80a108dae62 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:36.598226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 302/614] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:36.605282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae12d-3ae3-4c61-b624-fd66608c489e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:36.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae12d-3ae3-4c61-b624-fd66608c489e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:36.605845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae12d-3ae3-4c61-b624-fd66608c489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:36.613764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:36.634213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-649ae12d-3ae3-4c61-b624-fd66608c489e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:36.635856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-649ae12d-3ae3-4c61-b624-fd66608c489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:36.644905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:36.646675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:36.647347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:36.647793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:36.662307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:36.665606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:36.687229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:36.688341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:36.689322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:36.695308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 313/615] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:36.699949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:36.774889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 303/616] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:36.824628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:36.845330 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a17156-33bf-4e3e-b674-53524c506741 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:36.879828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:36.880752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:36.881242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:36.881710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:36.882627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:36.883111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:36.884716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:36.903763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 314/617] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 14:44:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:36.910305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ae12d-3ae3-4c61-b624-fd66608c489e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.051907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ae12d-3ae3-4c61-b624-fd66608c489e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:37.096194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.104393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:37.105036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.105336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.105605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.109139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:37.127204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:37.128961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:37.155048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:37.156019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:37.156895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:37.156895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:37.156895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:37.166809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 304/618] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:37.172158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.278830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abab6c7-e4c3-4f8f-a3f0-5b8fbc5e739c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:37.299129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fd730-5aec-44a3-8f70-a2e086409f51 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:37.314428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:37.315329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.315855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.316345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.317187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:37.317712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:37.319286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:37.332769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 305/619] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:37.339681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:37.340349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.340628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.340897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.343048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.350709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:37.355240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:37.360754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:37.361809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:37.362630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:37.366838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 315/620] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:37.372937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.436476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:37.511164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228766b-f3e9-4ab0-9f4e-eb093f62c789 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:37.522696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869289bf-055e-44a9-b0a7-ed0d8d740cd7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:37.552784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f3ec3-86fe-40ad-b83a-25efe6139baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:37.553866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3ec3-86fe-40ad-b83a-25efe6139baf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.554869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3ec3-86fe-40ad-b83a-25efe6139baf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.555317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3ec3-86fe-40ad-b83a-25efe6139baf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.562510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f2e0d-b0cf-4a86-b92d-cbe1145ec05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:37.562850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f3ec3-86fe-40ad-b83a-25efe6139baf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:37.563270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f2e0d-b0cf-4a86-b92d-cbe1145ec05c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.563622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f2e0d-b0cf-4a86-b92d-cbe1145ec05c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.563966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f2e0d-b0cf-4a86-b92d-cbe1145ec05c None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.564697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f2e0d-b0cf-4a86-b92d-cbe1145ec05c None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:37.565069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f2e0d-b0cf-4a86-b92d-cbe1145ec05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:37.565534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f3ec3-86fe-40ad-b83a-25efe6139baf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:37.566658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f2e0d-b0cf-4a86-b92d-cbe1145ec05c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:37.581193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 316/621] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:37.581193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cda20d-ffb4-46a4-9ffd-9d3913936874 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.581193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cda20d-ffb4-46a4-9ffd-9d3913936874 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.581193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cda20d-ffb4-46a4-9ffd-9d3913936874 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.581193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 317/622] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9b1f3ec3-86fe-40ad-b83a-25efe6139baf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:37.583985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:37.584867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:37.585849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:37.586291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cda20d-ffb4-46a4-9ffd-9d3913936874 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.588517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 306/623] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:37.590170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93a00b-a84c-4ea1-ad9d-7a01b88b85bf None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.590633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93a00b-a84c-4ea1-ad9d-7a01b88b85bf None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.591101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93a00b-a84c-4ea1-ad9d-7a01b88b85bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.610612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6d93a00b-a84c-4ea1-ad9d-7a01b88b85bf None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:37.612098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6d93a00b-a84c-4ea1-ad9d-7a01b88b85bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:37.662835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:37.691782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 307/624] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:37.692913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93a00b-a84c-4ea1-ad9d-7a01b88b85bf None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.741544 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cda20d-ffb4-46a4-9ffd-9d3913936874 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:37.757010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:37.805831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:37.806747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.807202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.807755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.828578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:37.830439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:37.853443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:37.854430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:37.855300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:37.855300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:37.855300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:37.861527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 318/625] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:37.863156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:37.884928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93a00b-a84c-4ea1-ad9d-7a01b88b85bf None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:37.958551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:37.959520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:37.959975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:37.960476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:37.961452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:37.961953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:37.963666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:37.988654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 308/626] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:37.994709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.031395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbf14b-f61c-49f9-9ae2-e1f4d378adfd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.054338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.085144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462e3ec-a1ea-4461-84a4-5ef0bc96035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.086254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462e3ec-a1ea-4461-84a4-5ef0bc96035d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.086830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462e3ec-a1ea-4461-84a4-5ef0bc96035d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.086830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462e3ec-a1ea-4461-84a4-5ef0bc96035d None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.087420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462e3ec-a1ea-4461-84a4-5ef0bc96035d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:38.087800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462e3ec-a1ea-4461-84a4-5ef0bc96035d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:38.089090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462e3ec-a1ea-4461-84a4-5ef0bc96035d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:38.100792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 319/627] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:38.102067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40b837-a5c1-48db-ad43-c77817a8cb3e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.102349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40b837-a5c1-48db-ad43-c77817a8cb3e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.102634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40b837-a5c1-48db-ad43-c77817a8cb3e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.107777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 320/628] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:38.107777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40b837-a5c1-48db-ad43-c77817a8cb3e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.140010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.147476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b325bd-bcc6-410b-b0ce-c83e83bb3097 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.187092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.187288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.187475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.187742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/552b3ea5ceab4114908b8108d415c9cd/roles/3a9af0f0d1d04401a0fee6860911b19f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.202991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=552b3ea5ceab4114908b8108d415c9cd, role_id=3a9af0f0d1d04401a0fee6860911b19f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:38.204414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:38.210546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40b837-a5c1-48db-ad43-c77817a8cb3e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.218772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 309/629] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:37 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/552b3ea5ceab4114908b8108d415c9cd/roles/3a9af0f0d1d04401a0fee6860911b19f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:44:38.219800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.259529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e15f3-17c9-4ddc-97ad-031fd747a64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.260278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e15f3-17c9-4ddc-97ad-031fd747a64d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.260604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e15f3-17c9-4ddc-97ad-031fd747a64d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.260875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e15f3-17c9-4ddc-97ad-031fd747a64d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.274144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e15f3-17c9-4ddc-97ad-031fd747a64d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-769e15f3-17c9-4ddc-97ad-031fd747a64d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:38.280038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:38.281138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:38.282017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 321/630] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:38.283295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7988a-c849-4129-aba7-f5f36d27c3a2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.287439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7988a-c849-4129-aba7-f5f36d27c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.287439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7988a-c849-4129-aba7-f5f36d27c3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.305606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-55f7988a-c849-4129-aba7-f5f36d27c3a2 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:38.307035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-55f7988a-c849-4129-aba7-f5f36d27c3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:38.343441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.355626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1f879-3545-4680-bdd5-15aa551a00e1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.397151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 322/631] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:38.397759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.398804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7988a-c849-4129-aba7-f5f36d27c3a2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.399264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.399686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.400113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] PATH_INFO: `/v3/users/glance` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.408244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:38.410515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:38.423953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:38.425120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 14:44:38.426263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:38.428596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 310/632] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:38.433887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.444645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.521629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7988a-c849-4129-aba7-f5f36d27c3a2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.557380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c54866-10ad-4408-aac3-b83af7e309fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.558026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c54866-10ad-4408-aac3-b83af7e309fc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.558262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c54866-10ad-4408-aac3-b83af7e309fc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.558529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c54866-10ad-4408-aac3-b83af7e309fc None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.559181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c54866-10ad-4408-aac3-b83af7e309fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:38.559476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c54866-10ad-4408-aac3-b83af7e309fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:38.560833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c54866-10ad-4408-aac3-b83af7e309fc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:38.566351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b02a5-360d-4784-9d01-3843def35aad None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.570433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 323/633] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:38.571717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7f3b-610d-4390-aeaf-6d28c88bc25b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.572000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7f3b-610d-4390-aeaf-6d28c88bc25b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.572272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7f3b-610d-4390-aeaf-6d28c88bc25b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.573489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 324/634] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:38.578673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f7f3b-610d-4390-aeaf-6d28c88bc25b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.609373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.613558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bae28-e101-4721-a744-f210e94b3235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.614846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bae28-e101-4721-a744-f210e94b3235 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.615728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bae28-e101-4721-a744-f210e94b3235 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.616083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bae28-e101-4721-a744-f210e94b3235 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.616903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bae28-e101-4721-a744-f210e94b3235 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:38.617337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bae28-e101-4721-a744-f210e94b3235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:38.618833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bae28-e101-4721-a744-f210e94b3235 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:38.639038 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 311/635] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 14:44:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:38.640618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747125f-9c00-4f40-97b9-195f413a22ab None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.640968 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747125f-9c00-4f40-97b9-195f413a22ab None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.641327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747125f-9c00-4f40-97b9-195f413a22ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.662468 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7747125f-9c00-4f40-97b9-195f413a22ab None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:38.663944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7747125f-9c00-4f40-97b9-195f413a22ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:38.703604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f7f3b-610d-4390-aeaf-6d28c88bc25b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.727132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.745224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 312/636] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:38.745224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747125f-9c00-4f40-97b9-195f413a22ab None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.748294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.749013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.749403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.749744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.762370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:38.763829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:38.779993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:38.780969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:38.781887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:38.781887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:38.781887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:38.785419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 325/637] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:38.786311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:38.866326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747125f-9c00-4f40-97b9-195f413a22ab None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.886696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.907021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.907631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.907963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.908232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.916918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6391b06-e9f3-4ca9-93ff-d3ef56b4c705 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:38.963623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:38.964876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:38.965438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:38.966291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:38.978164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:38.981798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:38.987826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:38.989150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:38.990110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:38.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:38.991564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.000197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 326/638] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:39.001631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.110923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 313/639] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:39.112009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.122770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192b8d4-79f5-438d-9a13-6d7d272fb5e0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.159455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.160055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.160329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.160598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.161245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:39.161546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.162912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.169365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.176052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 327/640] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:39.177273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.218742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.226720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76bc38-c25b-4b43-95b5-71b74faf9b26 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.263895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.264490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.264761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.265026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.265701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:39.266021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.267459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.281248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 314/641] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:39.281248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.284197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdac2b-1581-4e99-9b25-5c6b93ddec8b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.323013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.323782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.324065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.324337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.334646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:39.340438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:39.345451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 328/642] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:39.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.389013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0d0b0-8568-4879-a03c-bd3fe3e31712 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.412645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.424720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.425514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.425796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.426071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.438833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.440257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:39.455531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:39.456540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:39.457477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:39.457477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:39.457477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:39.457477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 315/643] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:39.457477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.464109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2938ac8-cf77-4b8f-865b-91ac054ad3c2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.472780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.500201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.501024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.501441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.501778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.510048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.511925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:39.524252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:39.525425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 329/644] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:39.526503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.571248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca3cb-6fb1-4454-8f3f-eba88e680ea6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.607625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.608257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.608536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.608806 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.619441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.621436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.635654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689d31f-34c2-416a-9054-b3d196184aed None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:39.636999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:39.637928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:39.639576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:39.639576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:39.639576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:39.639576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 316/645] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:39.639576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.658619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.675566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.676168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.676442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.676711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.677394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:39.677693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.679073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.686953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 330/646] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:39.687727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.706054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.752162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08c157-e3dc-4f08-8ebf-13981e4f3803 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.788141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.788555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7827ff-730d-423d-9abd-d5029a49a167 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.789264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.789733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.790290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.791221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:39.791518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.792877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.809987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 317/647] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:39.809987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.826290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.826892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.827168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.827436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.828142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:39.828440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.829886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.851332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 331/648] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:39.852221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.892355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.915476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41089d5-28f2-4b96-9d1f-6cf63bfd4b68 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.938517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:39.955670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.956273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.956548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.956854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:39.957074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9291ff-1585-48d0-8726-c57ec3b419e7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:39.957517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:39.957815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:39.959393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:39.972869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 318/649] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:39.973858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:39.997553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:39.998281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:39.998634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:39.998913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.014183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.016349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:40.030349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:40.031321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:40.032997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:40.032997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:40.032997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:40.032997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 332/650] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:40.032997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.088904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08469-a885-423a-8f13-72a6fad7b38c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.123826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.124516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.124729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.124998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.127116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.135363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.137633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.139705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74a1d-0755-4d4d-9ada-cd84ab320354 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:40.149260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:40.150105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:40.150988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 319/651] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:40.152040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.178184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.178788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.179065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.179882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.196251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.251353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.253117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.262368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f16db-4b67-4850-bd7f-a7b1b82eba0e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.297048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.297726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.297995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.298225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.307295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.308841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:40.320169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:40.321400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 320/652] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:40.322352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.350765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.368904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 333/653] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:40.368904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.432739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.436064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1700b-8728-4f18-854b-2b747a1a4b57 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.470626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23bbd2-1f24-4e90-abd5-17b410d94024 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.470879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.471489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.471767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.472046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.472677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:40.472976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.474358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.486893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 321/654] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:40.487673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.505615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.506212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.506496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.506769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.507475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:40.507778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.509234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.520369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 334/655] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:40.521141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.571969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.585168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419b225-26b1-4e8d-a908-f22bd66ef839 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.619904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4373f-8b90-4e90-8612-2ca759976192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.620521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4373f-8b90-4e90-8612-2ca759976192 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.620796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4373f-8b90-4e90-8612-2ca759976192 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.621061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4373f-8b90-4e90-8612-2ca759976192 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.621798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4373f-8b90-4e90-8612-2ca759976192 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:40.621965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dccb3-cb19-472e-8f05-308e741771fa None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.622094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4373f-8b90-4e90-8612-2ca759976192 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.623554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4373f-8b90-4e90-8612-2ca759976192 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.640323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 322/656] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:40.641571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811bbb-4453-4ab1-befa-c29efbda167a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.641854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811bbb-4453-4ab1-befa-c29efbda167a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.642121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811bbb-4453-4ab1-befa-c29efbda167a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.643456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 323/657] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:40.644036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53811bbb-4453-4ab1-befa-c29efbda167a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.656633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.657294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.657568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.657832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.663124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.669316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.670797 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:40.684354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:40.685335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:40.690969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:40.690969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:40.690969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:40.690969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 335/658] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:40.690969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.745673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53811bbb-4453-4ab1-befa-c29efbda167a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.778317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.778935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.779210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.779483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.782246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ad760-4fd8-4895-981a-b305fe63d1cf None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.783722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.786541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.787969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:40.799885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:40.800895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 324/659] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:40.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.822559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73423b83-b207-4589-bdb1-5c1e6ac481c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.823204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73423b83-b207-4589-bdb1-5c1e6ac481c4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.823481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73423b83-b207-4589-bdb1-5c1e6ac481c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.823750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73423b83-b207-4589-bdb1-5c1e6ac481c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.824460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73423b83-b207-4589-bdb1-5c1e6ac481c4 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=04d1fc63b356499d80061e358fb447e4) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:40.824760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73423b83-b207-4589-bdb1-5c1e6ac481c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:40.826217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73423b83-b207-4589-bdb1-5c1e6ac481c4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:40.831834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 336/660] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/role_assignments?user.id=04d1fc63b356499d80061e358fb447e4&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:40.832992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72528bd3-e649-4529-a8e6-b1c4ab22ce95 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.833301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72528bd3-e649-4529-a8e6-b1c4ab22ce95 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.833572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72528bd3-e649-4529-a8e6-b1c4ab22ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:40.853190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-72528bd3-e649-4529-a8e6-b1c4ab22ce95 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:40.854443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-72528bd3-e649-4529-a8e6-b1c4ab22ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:40.884381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:40.910769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504ac7c-77e4-4542-a001-3ae996d44f23 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:40.933420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 337/661] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:40.933420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72528bd3-e649-4529-a8e6-b1c4ab22ce95 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:40.961736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:40.962697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:40.962818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:40.963130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.051820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.063218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72528bd3-e649-4529-a8e6-b1c4ab22ce95 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.064612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.066097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.102444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.103053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.103333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.103604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.104244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:41.104546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.105969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.121112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 338/662] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:41.122263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.157482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.184629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 325/663] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:41.186316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.248408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5132e-690a-444d-92d3-8e94a8024943 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.282896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.283501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.283786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.284054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.284775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:41.285077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.286547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.293255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3c16-c65b-42a8-9af5-a19eaa88064c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.305871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 339/664] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:41.306975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.323009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.333036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.333700 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.333982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.334249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.341512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.342995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:41.355001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:41.355853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:41.356736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 326/665] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:41.358137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.402775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.414148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d1404-6f68-4474-8a05-d535b617ee5e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.449388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.450057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.450357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.450579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.461383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.462829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.472279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1937f40-1237-44d2-a061-87e9ded88677 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:41.477284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:41.478189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:41.479061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:41.479061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:41.479061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:41.479061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 340/666] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:41.479292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.516716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.518495 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.518967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.519439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.531285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:41.537564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:41.538722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:41.539741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:41.540843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 327/667] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:41.541682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.555310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.591407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87214-6fdf-4904-b240-bc07224c06d0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.626788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.627392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.627680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.627930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.651501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.655803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ffdb3-57cb-4435-afdc-d3be9779f435 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.691278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.692119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.692606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.693011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.693873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:41.694321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.695937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.701003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.702473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.709065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 328/668] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:41.710079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.772262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.815282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 341/669] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:41.815282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.827014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c3464-14d6-49d3-8d6e-eedd599560da None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.864524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.865125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.865445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.865716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.866346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:41.866698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.868543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.880388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 329/670] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:41.881174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.902695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:41.919017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbb4e9-9685-49d3-8a86-9eb833a921e8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.952092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:41.952713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:41.952988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:41.953285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:41.953933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:41.954280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:41.955651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:41.963538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 342/671] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:41.968789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:41.987824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319a97c-ef18-4eb1-bae2-6d0996c62aa8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:41.997828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.030597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.031562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.032105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.032382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.033225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=7b6809bf91d244e4bed5b8a256a3f95a) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:42.033545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.035085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.053473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 330/672] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/role_assignments?user.id=7b6809bf91d244e4bed5b8a256a3f95a&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:42.053473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.100540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d850-78d8-4e55-be4d-e58e9c9df98a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.139928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.140729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.141151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.141577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.152314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.159650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.162868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.174715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a44cff-a997-4214-89a3-3f688cefbe52 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:42.184248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:42.185435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:42.186302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:42.186302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:42.186302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:42.189393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 343/673] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:42.190369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.221548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.222267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.222543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.222810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.232111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.233852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:42.249663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:42.250592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:42.251513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:42.252020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 331/674] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:42.252486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.296026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.321137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a13cd-1a79-4d69-9245-b8d531dad434 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.375832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.376738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.377159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.377671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.378946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c023eaa-fe89-4aa8-ab7f-3d14416c1dcd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.390244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.391893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:42.409263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:42.410249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:42.411137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:42.411137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:42.411137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:42.416425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 344/675] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:42.417255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.421937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.423320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.423320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.423320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.423820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:42.424178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.425645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.438755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 332/676] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:42.439640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.457554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.547391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bae2c-6050-4188-aa51-9c6596415f73 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.554675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1c23b-7920-4c99-a353-ceb3fbb1f7cb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.584098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.590172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.591147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.591647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.592328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.593398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:42.593806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.595434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.602573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.603496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.603760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.604029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.604665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:42.604963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.606443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.615746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 345/677] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:42.615746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.627563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 333/678] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:42.628668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.705675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.736515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6ea5f-f6f4-4b94-bdc4-457e83c54d80 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.748942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414dd9c-1f51-4539-9f51-1ffb4cf9e9a8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.774221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.774880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.775223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.775497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.784697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.785665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:42.786270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:42.786377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.786552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:42.786809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:42.787526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=b46482f26051455daeed2c00f8ced4bd) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:42.787830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:42.790298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:42.796651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 334/679] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/role_assignments?user.id=b46482f26051455daeed2c00f8ced4bd&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:42.797282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:42.801592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:42.802510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:42.803369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:42.805555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 346/680] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:42.809367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:42.888901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.937462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb883a-5e32-48c1-a32c-43fc9c208946 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:42.982921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:42.992369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9e8e3-ab30-4c4a-afd0-584a39be1cb1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:43.009723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:43.010379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.010653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.010920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.025916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:43.025916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:43.046373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:43.047467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:43.048388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:43.059753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 335/681] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:43.061811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:43.087807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:43.088426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.088705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.088971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.089737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:43.090038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:43.091591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:43.109427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 347/682] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:43.115150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:43.186681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55443411-6f5d-45a6-b55c-2b9af60bb818 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:43.232315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a51f9a-5e35-432d-b2ec-154d1409c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:43.232933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51f9a-5e35-432d-b2ec-154d1409c85c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.233230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51f9a-5e35-432d-b2ec-154d1409c85c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.233525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51f9a-5e35-432d-b2ec-154d1409c85c None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.234522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a51f9a-5e35-432d-b2ec-154d1409c85c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:43.234819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a51f9a-5e35-432d-b2ec-154d1409c85c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:43.236969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a51f9a-5e35-432d-b2ec-154d1409c85c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:43.241859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1105f-8375-4b6f-b843-1f037cd4d506 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:43.256079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:43.261293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 336/683] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 14:44:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:43.262865 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604a5d-b92b-4d46-8815-405d0b6adbd1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.263158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604a5d-b92b-4d46-8815-405d0b6adbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.263435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604a5d-b92b-4d46-8815-405d0b6adbd1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.264775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 337/684] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:43.270300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06604a5d-b92b-4d46-8815-405d0b6adbd1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:43.284320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b3cff-2d44-4def-b503-722c08883526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:43.284920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3cff-2d44-4def-b503-722c08883526 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.285293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3cff-2d44-4def-b503-722c08883526 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.285648 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3cff-2d44-4def-b503-722c08883526 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.286429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b3cff-2d44-4def-b503-722c08883526 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=f9edd8cf448e42fd8e888c8f531fa70d) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:43.286802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b3cff-2d44-4def-b503-722c08883526 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:43.288463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b3cff-2d44-4def-b503-722c08883526 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:43.295404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 348/685] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:43 2022] GET /identity/v3/role_assignments?user.id=f9edd8cf448e42fd8e888c8f531fa70d&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:43.296885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439e4fe-5269-4634-8a47-66ce88693331 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.297309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439e4fe-5269-4634-8a47-66ce88693331 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.297649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439e4fe-5269-4634-8a47-66ce88693331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.317913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4439e4fe-5269-4634-8a47-66ce88693331 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:43.320643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4439e4fe-5269-4634-8a47-66ce88693331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:43.336886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:43.405176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06604a5d-b92b-4d46-8815-405d0b6adbd1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:43.417079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 349/686] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:43.446404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:43.447306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.447782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.448192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.550011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:43.551715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:43.608950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:43.666135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 338/687] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 14:44:43.671580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439e4fe-5269-4634-8a47-66ce88693331 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:43.751068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:43.781345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439e4fe-5269-4634-8a47-66ce88693331 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:43.792247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:43.824782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:43.825558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.825921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.826300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.843646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:43.845494 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:43.868342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:43.869374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:43.870257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:43.870257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:43.870257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:43.872803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 350/688] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:43.877970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:43.905688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f125c7-b5a2-4fb2-b00b-56a0c62ac1ce None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:43.969661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:43.970360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:43.970638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:43.970904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:43.989117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:43.990925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:44.002169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:44.028917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:44.029925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:44.030788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:44.030788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:44.030788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:44.035493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 339/689] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:44.042758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:44.067154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf1f15-c6b0-4f93-989e-d5f967335577 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:44.112778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:44.113429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.113706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.113974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.114611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:44.114907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:44.116289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:44.128824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 351/690] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:44.138421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:44.196712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36898b-c967-400b-ad1d-ca8b4b110fcb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:44.201522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:44.266371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:44.267331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.267831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.268353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.269323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:44.269850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:44.271409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:44.297056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 340/691] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:44.302886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:44.334311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63695854-0210-4115-8ed5-2640dc3d08ae None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:44.400496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:44.401050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.401356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.401629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.416324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:44.417237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:44.418726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:44.441671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:44.442687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:44.444210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 352/692] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:44.451410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:44.479333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e64f52-660d-4a6a-b660-b53b48624038 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:44.514275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:44.514891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.515165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.515434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.516084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:44.517533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:44.556164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 341/693] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:44:44 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:44:44.569683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:44.597110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8413e57-e975-42ee-86a3-af8958632fd3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:44.636058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ad37e-53e7-4ba9-9315-2ff83e8c68cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:44.636976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ad37e-53e7-4ba9-9315-2ff83e8c68cc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.637287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ad37e-53e7-4ba9-9315-2ff83e8c68cc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.637563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ad37e-53e7-4ba9-9315-2ff83e8c68cc None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.638260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ad37e-53e7-4ba9-9315-2ff83e8c68cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:44.638566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ad37e-53e7-4ba9-9315-2ff83e8c68cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:44.640160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ad37e-53e7-4ba9-9315-2ff83e8c68cc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:44.654640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 353/694] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:44.660077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:44.707333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe99f-e5af-4c7a-b33e-6ad87e38b1ca None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.707665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe99f-e5af-4c7a-b33e-6ad87e38b1ca None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.707953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe99f-e5af-4c7a-b33e-6ad87e38b1ca None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.709425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 354/695] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:44.716891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dda19e-b50d-4ab5-8d53-767fcedb80db None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.717581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dda19e-b50d-4ab5-8d53-767fcedb80db None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.718077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dda19e-b50d-4ab5-8d53-767fcedb80db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.738865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f5dda19e-b50d-4ab5-8d53-767fcedb80db None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:44.740427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f5dda19e-b50d-4ab5-8d53-767fcedb80db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:44.824323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 355/696] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:44.852594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00742154-9bef-4f87-af44-e5056b8c0541 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:44.866040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:44.894496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:44.895291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:44.896076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:44.896745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:44.898488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=552b3ea5ceab4114908b8108d415c9cd) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:44.898798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:44.900333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:44.907063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 342/697] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:44 2022] GET /identity/v3/role_assignments?user.id=552b3ea5ceab4114908b8108d415c9cd&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:45.118432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dda19e-b50d-4ab5-8d53-767fcedb80db None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:45.186698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:45.225958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dda19e-b50d-4ab5-8d53-767fcedb80db None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:45.264693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:45.265371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:45.265650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:45.265920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:45.276523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:45.281336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:45.282271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:45.283110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 356/698] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:45.288499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:45.421815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a1357-204f-4b52-b79f-ec42e48d2394 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:45.470227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acc0f6-cf64-4da7-8f52-32de9b49cea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:45.470769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acc0f6-cf64-4da7-8f52-32de9b49cea9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:45.471049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acc0f6-cf64-4da7-8f52-32de9b49cea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:45.471320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acc0f6-cf64-4da7-8f52-32de9b49cea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:45.471980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acc0f6-cf64-4da7-8f52-32de9b49cea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:45.472299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acc0f6-cf64-4da7-8f52-32de9b49cea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:45.473675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acc0f6-cf64-4da7-8f52-32de9b49cea9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:45.486767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 343/699] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:45.494272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:45.649003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7eac3-ad45-4cc9-b1f4-12e7c9bb905b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:45.684803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:45.686617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21024f2-f165-44b2-b4e1-6fc576131fb9 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:45.687462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21024f2-f165-44b2-b4e1-6fc576131fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:45.687939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21024f2-f165-44b2-b4e1-6fc576131fb9 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:45.689987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 344/700] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:45.693267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:45.693761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:45.694043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:45.694318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:45.697411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c81743-7b1e-4fc8-917a-b703f071872b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:45.697925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c81743-7b1e-4fc8-917a-b703f071872b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:45.698615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c81743-7b1e-4fc8-917a-b703f071872b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:45.713298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:45.714767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:45.720315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a0c81743-7b1e-4fc8-917a-b703f071872b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:45.721798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a0c81743-7b1e-4fc8-917a-b703f071872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:45.742698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:45.743728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:45.744665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:45.744665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:45.744665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:45.744665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 357/701] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:45.758751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:45.761881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:45.866553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 345/702] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:45.868579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a119843-28ce-4ee0-9b45-60f24564abba None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:45.908140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074c4f3-dacb-4ed4-9763-90ddd6b4e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:45.908772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074c4f3-dacb-4ed4-9763-90ddd6b4e44f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:45.909111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074c4f3-dacb-4ed4-9763-90ddd6b4e44f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:45.909458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074c4f3-dacb-4ed4-9763-90ddd6b4e44f None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:45.910177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074c4f3-dacb-4ed4-9763-90ddd6b4e44f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:45.910535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074c4f3-dacb-4ed4-9763-90ddd6b4e44f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:45.911948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074c4f3-dacb-4ed4-9763-90ddd6b4e44f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:45.926002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 358/703] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:45.931127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c81743-7b1e-4fc8-917a-b703f071872b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.097388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c81743-7b1e-4fc8-917a-b703f071872b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:46.132770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb9931-a230-4799-b7eb-52b90f64c9fe None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.133162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb9931-a230-4799-b7eb-52b90f64c9fe None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.133460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb9931-a230-4799-b7eb-52b90f64c9fe None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.135217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 359/704] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:46.146812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0357-ca56-44ad-9c23-0cc009d7da5d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.147219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0357-ca56-44ad-9c23-0cc009d7da5d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.147574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:46.147739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0357-ca56-44ad-9c23-0cc009d7da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.149027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.149479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.149879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.157712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:46.159412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:46.173100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:46.174079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:46.174997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:46.177404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 346/705] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:44:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:46.182496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.186241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-605a0357-ca56-44ad-9c23-0cc009d7da5d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:46.187444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-605a0357-ca56-44ad-9c23-0cc009d7da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:46.215178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:46.237513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:46.289448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 360/706] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:46.289448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a0357-ca56-44ad-9c23-0cc009d7da5d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.316267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828fed5-b07f-45b7-8190-08a395e1c560 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:46.372707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:46.373459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.373746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.374018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.374771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:46.375076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:46.376793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:46.397634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 347/707] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:44:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:46.404035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.420111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a0357-ca56-44ad-9c23-0cc009d7da5d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:46.466063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:46.467060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.468266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.468866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.469860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:46.472799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:46.490428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:46.492048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:46.492889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:46.495808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 361/708] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 9 14:44:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:46.501496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.557398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:46.573983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:46.590831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58483b6a-b65d-40b3-8e3d-07a19222aa64 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:46.648620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:46.649880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.650368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.650864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.666047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09d263-4fce-4502-999f-37813a0b4b8c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:46.701573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:46.702434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.702434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.702716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.703331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:46.703630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:46.705024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:46.709961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 362/709] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 14:44:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:46.710804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.728713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:46.730980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:46.824152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:46.828416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56184ee-dd1e-4c57-b0c0-1fda718a0760 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:46.835673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:46.856574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 348/710] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 14:44:46.857559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.868234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:46.869151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:46.870121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:46.870619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] PATH_INFO: `/v3/services/network` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:46.871402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:46.873038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:46.883323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:46.884369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:46.885303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:46.888262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 363/711] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:46 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:46.889374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:46.971773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f0906-1eb5-4a53-b2cd-fb1149fded0c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:46.995397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb1840-1547-4ceb-81f2-6b262cf2128f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:47.017108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:47.021460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.021972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.022453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.023368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:47.023935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:47.026327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:47.034925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 349/712] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 14:44:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:47.036070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:47.050135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:47.051555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.051845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.052117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.069165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:47.071248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:47.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:47.089673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:47.090650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:47.090650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:47.090650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:47.090650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 364/713] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:47.095183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:47.124696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:47.162715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:47.231363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b66f22-b442-4130-8b84-085162c5a0ae None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:47.277060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:47.277716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.277992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.278263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.278919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:47.279215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:47.280618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:47.295746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f59a1-96c9-4958-8bd1-52e0ded1d4e6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:47.304810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 365/714] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:47.310552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:47.359922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:47.360458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.360736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.361009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.361697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] RBAC: Adding query filter params (name=network) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:47.361999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:47.363443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:47.368489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 350/715] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:44:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:47.378297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:47.399327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:47.434473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb1a50-9f0a-4f2b-8932-2df15bc101cd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:47.473021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea950c-0189-40e0-bb31-83f403292e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:47.473736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea950c-0189-40e0-bb31-83f403292e54 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.474115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea950c-0189-40e0-bb31-83f403292e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.474524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea950c-0189-40e0-bb31-83f403292e54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.482430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea950c-0189-40e0-bb31-83f403292e54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:47.484162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea950c-0189-40e0-bb31-83f403292e54 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2dea950c-0189-40e0-bb31-83f403292e54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:47.496484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:47.498591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 366/716] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d22d4-64eb-4e89-8272-5872af44757d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.500700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d22d4-64eb-4e89-8272-5872af44757d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.506070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d22d4-64eb-4e89-8272-5872af44757d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.506070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 367/717] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:47.506634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d22d4-64eb-4e89-8272-5872af44757d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:47.533911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6096-f092-4962-84a7-951283f35c96 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:47.553333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:47.591118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0958-66cb-40b7-9367-53fceb7e35bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:47.592180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0958-66cb-40b7-9367-53fceb7e35bd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.593549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0958-66cb-40b7-9367-53fceb7e35bd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.594377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0958-66cb-40b7-9367-53fceb7e35bd None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.595445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0958-66cb-40b7-9367-53fceb7e35bd None admin] RBAC: Adding query filter params (type=network) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:47.596622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0958-66cb-40b7-9367-53fceb7e35bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:47.599996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0958-66cb-40b7-9367-53fceb7e35bd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:47.613630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 351/718] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:44:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:47.617507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b664-06f6-4a7c-8c19-a36d588c8edf None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.618082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b664-06f6-4a7c-8c19-a36d588c8edf None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.618580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b664-06f6-4a7c-8c19-a36d588c8edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.659387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9347b664-06f6-4a7c-8c19-a36d588c8edf None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:47.662272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9347b664-06f6-4a7c-8c19-a36d588c8edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:47.687311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d22d4-64eb-4e89-8272-5872af44757d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:47.782378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:47.783422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.783923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.784433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.785407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:47.785915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:47.787582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:47.792441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:47.816816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 368/719] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:47.824675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:47.837998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 352/720] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:47.839828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0492a84-59cf-4e1d-bc85-65354fbef573 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.840121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0492a84-59cf-4e1d-bc85-65354fbef573 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.840396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0492a84-59cf-4e1d-bc85-65354fbef573 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.849224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 353/721] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:47.849224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a275cb-5c59-4a96-928c-68ff187b21dc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:47.849224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a275cb-5c59-4a96-928c-68ff187b21dc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:47.849224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a275cb-5c59-4a96-928c-68ff187b21dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:47.899180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a8a275cb-5c59-4a96-928c-68ff187b21dc None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:47.900390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a8a275cb-5c59-4a96-928c-68ff187b21dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:48.064822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 354/722] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 14:44:48.067166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e020908-0dc9-468d-8769-398888b5396c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:48.129240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:48.129240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:48.129240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:48.129240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:48.129663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=7b6809bf91d244e4bed5b8a256a3f95a) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:48.129663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:48.130268 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:48.151245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 369/723] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:47 2022] GET /identity/v3/role_assignments?user.id=7b6809bf91d244e4bed5b8a256a3f95a&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:48.244168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a275cb-5c59-4a96-928c-68ff187b21dc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:48.302312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:48.383709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a275cb-5c59-4a96-928c-68ff187b21dc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:48.385639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:48.436907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:48.437533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:48.437806 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:48.438071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:48.457435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:48.462787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:48.463731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:48.465411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 355/724] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:48.470126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:48.490076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:48.579848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5339c0d-181e-438f-8e37-eedc9af1463a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:48.646796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:48.647216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:48.647442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:48.647742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:48.648015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:48.651593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c736bba-36e8-439b-a7e4-ffbd17e8fbd1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:48.670257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:48.675528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:48.676563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:48.677455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:48.685656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 370/725] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:48.691454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:48.702992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:48.703616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:48.703889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:48.704154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:48.704824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:48.705120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:48.706502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:48.724660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 356/726] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:48.733562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:48.870421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2b3d-1c92-42d9-ad4c-e861f77e9d0b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:48.901458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161dbee-e91d-4223-a9dd-1142f80ff63b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:48.913164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:48.914363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:48.914637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:48.914902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:48.933835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:48.935265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:48.936559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:48.961569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:48.962518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:48.963390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:48.963390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:48.963390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:48.964136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 357/727] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:48.968421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:48.982069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:48.982680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:48.982954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:48.983219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:48.983876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:48.984170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:48.985560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:48.994647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:49.000075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 371/728] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:49.008193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:49.109393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948ab88-0943-4caf-82b1-4fd8165092a0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:49.145333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:49.145915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.146193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.146465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.147106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:49.147411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:49.153238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:49.163497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 358/729] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:49.168403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:49.220309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec5462-5087-4f31-9686-152137da3a43 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:49.278597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:49.307295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:49.307940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.308216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.308485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.329780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4ed2-e1ba-4e10-8c75-62e34f5dba99 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:49.332135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:49.333614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:49.358244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:49.359202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:49.360507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:49.360507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:49.360507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:49.360507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 372/730] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:49.366325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:49.415105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:49.415105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.415105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.415105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.424954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:49.427038 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:49.428482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:49.453842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:49.454791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:44:49.455712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:49.462832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 359/731] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:44:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:49.466061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:49.560034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d893a27-2ad6-4ecd-b6f2-20076379bb44 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:49.600189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:49.600834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.601113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.601398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.602032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:49.602329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:49.604488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:49.622057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 373/732] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:49.630218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:49.656297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0e0b5-7a0a-460e-9275-9a7e9479a93c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:49.704929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:49.712005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e58f12-b438-4230-91b0-705459a3e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:49.712615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58f12-b438-4230-91b0-705459a3e193 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.712888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58f12-b438-4230-91b0-705459a3e193 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.713163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58f12-b438-4230-91b0-705459a3e193 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.713913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e58f12-b438-4230-91b0-705459a3e193 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:49.714217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e58f12-b438-4230-91b0-705459a3e193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:49.716476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e58f12-b438-4230-91b0-705459a3e193 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:49.739743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 360/733] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 14:44:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:49.741946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38608d13-3bbf-4e94-b7b8-e477db989271 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.742429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38608d13-3bbf-4e94-b7b8-e477db989271 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.743048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:49.744044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38608d13-3bbf-4e94-b7b8-e477db989271 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.745709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 361/734] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:49.750318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38608d13-3bbf-4e94-b7b8-e477db989271 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:49.751980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15353242-4b31-4e2e-9d88-01c69ad1be70 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:49.790105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4a2d5-2105-445d-aee4-579bfe291a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:49.790716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4a2d5-2105-445d-aee4-579bfe291a73 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.790999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4a2d5-2105-445d-aee4-579bfe291a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.791264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4a2d5-2105-445d-aee4-579bfe291a73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.798671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4a2d5-2105-445d-aee4-579bfe291a73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:49.800113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4a2d5-2105-445d-aee4-579bfe291a73 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2fa4a2d5-2105-445d-aee4-579bfe291a73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:49.812658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:49.813827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:49.814815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 374/735] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:49.815397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f4ce5-66bc-40e6-a48e-93e7f15b4552 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.815465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f4ce5-66bc-40e6-a48e-93e7f15b4552 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.815691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f4ce5-66bc-40e6-a48e-93e7f15b4552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.839481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bc1f4ce5-66bc-40e6-a48e-93e7f15b4552 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:49.840668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bc1f4ce5-66bc-40e6-a48e-93e7f15b4552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:49.877970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38608d13-3bbf-4e94-b7b8-e477db989271 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:49.919331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:49.920424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:49.921282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:49.922035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:49.937223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 375/736] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:49.937223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f4ce5-66bc-40e6-a48e-93e7f15b4552 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:50.017676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:50.027164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:50.028649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:50.113537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:50.153469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f4ce5-66bc-40e6-a48e-93e7f15b4552 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:50.193982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 362/737] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:50.195677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:50.246247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:50.247663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.248240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.248703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.249676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:50.250177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:50.251997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:50.271799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 376/738] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 14:44:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:50.273473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:50.300930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:50.312085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350641-4bd3-4a26-98bc-4e885a0f4338 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:50.349573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:50.350471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.351581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.352004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.363048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:50.364790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:50.380462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d0573-917f-4b2a-b7d1-12b828e3faaf None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:50.383590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:50.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:50.385808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:50.385808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:50.385808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:50.387729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 363/739] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:50.388750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:50.419437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c7bd8-73ee-42e6-9bfe-eceacfda61ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:50.420038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c7bd8-73ee-42e6-9bfe-eceacfda61ef None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.420320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c7bd8-73ee-42e6-9bfe-eceacfda61ef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.420588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c7bd8-73ee-42e6-9bfe-eceacfda61ef None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.421250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c7bd8-73ee-42e6-9bfe-eceacfda61ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:50.422608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c7bd8-73ee-42e6-9bfe-eceacfda61ef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:50.434560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:50.449223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 377/740] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:44:50 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:50.449223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d96eb-5521-4253-bbec-6bd9a5d4cd8d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.449223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d96eb-5521-4253-bbec-6bd9a5d4cd8d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.449223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d96eb-5521-4253-bbec-6bd9a5d4cd8d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.453402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 378/741] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:50.454376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d96eb-5521-4253-bbec-6bd9a5d4cd8d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:50.520427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006a0c-7e29-4c14-8423-58c7df4882d3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:50.575799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438fac2-dec2-4938-9811-5b57a6898814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:50.576436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438fac2-dec2-4938-9811-5b57a6898814 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.576712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438fac2-dec2-4938-9811-5b57a6898814 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.576977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438fac2-dec2-4938-9811-5b57a6898814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.587294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:50.591745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d96eb-5521-4253-bbec-6bd9a5d4cd8d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:50.632771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6530d-1db0-4cb3-a8d8-fa7f913ba640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:50.633797 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6530d-1db0-4cb3-a8d8-fa7f913ba640 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.634444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6530d-1db0-4cb3-a8d8-fa7f913ba640 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.635024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6530d-1db0-4cb3-a8d8-fa7f913ba640 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.635967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6530d-1db0-4cb3-a8d8-fa7f913ba640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:50.636736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6530d-1db0-4cb3-a8d8-fa7f913ba640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:50.638441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6530d-1db0-4cb3-a8d8-fa7f913ba640 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:50.651348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 379/742] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:50.652933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ac155-7c6f-4deb-9176-415f63eee131 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.654095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ac155-7c6f-4deb-9176-415f63eee131 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.654446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ac155-7c6f-4deb-9176-415f63eee131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.658531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438fac2-dec2-4938-9811-5b57a6898814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:50.659988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438fac2-dec2-4938-9811-5b57a6898814 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:50.672613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-537ac155-7c6f-4deb-9176-415f63eee131 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:50.673921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-537ac155-7c6f-4deb-9176-415f63eee131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:50.735803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:50.747490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 380/743] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:50.748780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693df9-f69f-4402-905e-c13ea0495979 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.749062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693df9-f69f-4402-905e-c13ea0495979 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.749362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693df9-f69f-4402-905e-c13ea0495979 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.753312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 381/744] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:50.753312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693df9-f69f-4402-905e-c13ea0495979 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:50.787366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 364/745] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:50 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 14:44:50.788895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d6ab9-33c3-4eaf-8c2e-dd6378fc25d0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.789287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d6ab9-33c3-4eaf-8c2e-dd6378fc25d0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.789691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d6ab9-33c3-4eaf-8c2e-dd6378fc25d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.811398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-880d6ab9-33c3-4eaf-8c2e-dd6378fc25d0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:50.812698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-880d6ab9-33c3-4eaf-8c2e-dd6378fc25d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:50.851436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:50.866801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693df9-f69f-4402-905e-c13ea0495979 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:50.897012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 365/746] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 14:44:50.899110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d6ab9-33c3-4eaf-8c2e-dd6378fc25d0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:50.912898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:50.913753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:50.914127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:50.914663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:50.928239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:50.929892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:50.953714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:50.954833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:50.955772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:50.962848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 382/747] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:50.962848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.028776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d6ab9-33c3-4eaf-8c2e-dd6378fc25d0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.067058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:51.075801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.076958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.077341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.077888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.089855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.091402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:51.109993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:51.111152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:51.112231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:51.112231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:51.112231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:51.112231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 366/748] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:51.112231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.133911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d545c8b-e9b9-4f14-9f41-14dc73755c44 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.142734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:51.169696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.170368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.170749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.171125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.171922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:51.172327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.173909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.187160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 383/749] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:51.188209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.236236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdff386-f9b6-4ba3-a763-559b70551f4c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.275767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.276381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.276663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.276961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.277935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.278980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.295681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 367/750] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:44:51 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:51.298099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.298802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db93dcc-2a9c-40a6-a25c-2f17d879abc7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.315968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:51.362718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.364026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.364515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.365299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.367346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:51.368175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.370711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.392909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 384/751] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:51.392909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.419963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:51.447527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3b370-706a-496f-86ef-5fc15796ba83 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.502910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.503542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.503827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.504103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.504832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=b3e63489796544afaa44deab302681b9) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:51.505142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.506714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.514442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 368/752] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/role_assignments?user.id=b3e63489796544afaa44deab302681b9&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:51.515121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.550827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4ce6-6ae9-4089-85b0-ba00f6f3ad85 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.591572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.592224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.592505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.592776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.606371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:51.612320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:51.613498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:51.614472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 385/753] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:51.620280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.651805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:51.671156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4cf59-36b2-4ace-b7b7-d4be86c1c2bb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.721151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.721824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.722105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.722379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.731234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.732706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:51.747373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:51.748391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:51.749434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 369/754] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:51.754286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.759088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:51.810825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71a1a0-ebb6-4880-b064-7d66e15f04d0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.878732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc929ccd-e7b7-4d85-b7b0-2e403c8588b8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:51.909899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.910962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.911478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.915737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.916458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:51.916773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.918193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.940598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:51.945219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:51.945357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 386/755] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:51.945439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:51.945439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:51.945439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:51.945439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:51.945439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:51.946337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:51.963465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 370/756] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:51.970199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:52.057222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:52.093394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2c20-4734-4b3d-b360-45bd9b91df92 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:52.129508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:52.130121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:52.130404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:52.130678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:52.131402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=552b3ea5ceab4114908b8108d415c9cd) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:52.131708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:52.133699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:52.140032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 371/757] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/role_assignments?user.id=552b3ea5ceab4114908b8108d415c9cd&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:52.144710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:52.175370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372fd06-d92b-43d7-ab0e-cdc425f1a274 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:52.244102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:52.244667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:52.244948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:52.245242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:52.257508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:52.259016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:52.392179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:52.393609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:52.394470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:52.394470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:52.394470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:52.394470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 387/758] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:52.400742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:52.518756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a37e52-3a03-4549-9ed3-b427d4c54b38 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:52.549618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:52.555826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:52.556611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:52.557172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:52.557475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:52.558173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:52.558488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:52.559850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:52.577464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 372/759] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:52.578849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:52.702483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bf5ea-26a2-4f19-9135-7a7cad5ebb35 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:52.749303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2fca2-0692-449b-a4db-1954e5c5014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:52.749952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2fca2-0692-449b-a4db-1954e5c5014f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:52.750236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2fca2-0692-449b-a4db-1954e5c5014f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:52.750509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2fca2-0692-449b-a4db-1954e5c5014f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:52.760168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2fca2-0692-449b-a4db-1954e5c5014f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:52.761907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2fca2-0692-449b-a4db-1954e5c5014f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:52.768181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-52b2fca2-0692-449b-a4db-1954e5c5014f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:52.778225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:52.779117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:52.780046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 388/760] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:44:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:52.785125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:52.833882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d307d-7de3-4cb8-a19e-3959ed5a5f1f None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:52.834239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d307d-7de3-4cb8-a19e-3959ed5a5f1f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:52.834484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d307d-7de3-4cb8-a19e-3959ed5a5f1f None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:52.836707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 389/761] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:52.841920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e2aa9-0ed5-4136-9fed-e469ecd639d7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:52.842202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e2aa9-0ed5-4136-9fed-e469ecd639d7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:52.842478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e2aa9-0ed5-4136-9fed-e469ecd639d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:52.861397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-808e2aa9-0ed5-4136-9fed-e469ecd639d7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:52.862630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-808e2aa9-0ed5-4136-9fed-e469ecd639d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:52.897600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59141c2-43e8-402b-bd06-4c142f16d5bf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:52.936056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:52.937046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:52.937543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:52.938167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:52.939183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:52.939773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:52.941432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:52.961119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 373/762] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:44:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:52.967089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:52.987834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 390/763] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:44:53.027489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:53.078236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3169b8-b70e-4a81-8d8f-9ccfbc994f61 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:53.112816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:53.113818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:53.114294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:53.114834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:53.125752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:53.127439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:53.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:53.143535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:53.144420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:53.144420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:53.144420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:53.146506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 374/764] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:53.154323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e2aa9-0ed5-4136-9fed-e469ecd639d7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:53.302795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e2aa9-0ed5-4136-9fed-e469ecd639d7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:53.331588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:53.346487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:53.347112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:53.347389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:53.347659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:53.348309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:53.348610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:53.350049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:53.369263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 391/765] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:53.374863 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:53.474062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:53.493914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5be966-9c98-449f-bd6d-a1a0081bc739 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:53.496201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:53.532750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:53.533762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:53.534294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:53.534985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:53.543144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:53.544885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:53.557071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:53.557995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:53.558875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:53.563192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 375/766] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:53.573221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:53.686606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f389c2-5ac8-419a-8713-a81ec3947770 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:53.694013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99923920-9774-422f-8c26-7fd1e451067c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:53.751159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:53.752082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:53.752672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:53.753047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:53.753880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:53.754296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:53.755882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:53.760737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:53.761406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:53.761937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:53.762205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:53.762787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:53.764156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:53.782511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 376/767] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:53.783736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:53.784690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:53.785560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:53.789663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 392/768] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 9 14:44:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:53.795352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:53.796566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:53.838215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:53.907888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:53.973820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ac701-5278-4f77-a712-aea9f0ac2dd3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:53.992865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6280b-3a18-4507-b097-2eb27b103de3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:54.061746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727a4c5-8658-457a-ab44-25b44f035328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:54.062401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:54.063021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.063300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.063574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.064209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:54.064515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:54.065915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:54.067490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727a4c5-8658-457a-ab44-25b44f035328 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.067767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727a4c5-8658-457a-ab44-25b44f035328 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.068036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727a4c5-8658-457a-ab44-25b44f035328 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/7b6809bf91d244e4bed5b8a256a3f95a/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.070603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727a4c5-8658-457a-ab44-25b44f035328 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=7b6809bf91d244e4bed5b8a256a3f95a, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:54.072776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727a4c5-8658-457a-ab44-25b44f035328 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:54.078164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 377/769] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 14:44:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:54.087006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:54.094407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 393/770] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:53 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/7b6809bf91d244e4bed5b8a256a3f95a/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:54.204539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fedd0-0b64-400a-8a3f-ce2f7e2868a1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.204889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fedd0-0b64-400a-8a3f-ce2f7e2868a1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.205093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fedd0-0b64-400a-8a3f-ce2f7e2868a1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.206844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 394/771] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:44:54.215238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca614c7-561f-4158-84e8-666536a97000 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.215873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca614c7-561f-4158-84e8-666536a97000 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.216344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca614c7-561f-4158-84e8-666536a97000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.235004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:54.248417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e73095-1391-4a46-b84b-ac2a0bbaf54b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:54.265702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5ca614c7-561f-4158-84e8-666536a97000 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:54.267283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5ca614c7-561f-4158-84e8-666536a97000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:54.286498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:54.287109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.287385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.287652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.288328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:54.288629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:54.290029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:54.296094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 378/772] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 14:44:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:54.301814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:54.385807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 395/773] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:44:54.387397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5e8a-643d-401c-8e2b-d937b6a240a4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.387672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5e8a-643d-401c-8e2b-d937b6a240a4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.387937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5e8a-643d-401c-8e2b-d937b6a240a4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.389391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 396/774] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:54.396006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07d505-4fcf-4e02-83e3-1b9b7553ecd0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.396674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07d505-4fcf-4e02-83e3-1b9b7553ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.397043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07d505-4fcf-4e02-83e3-1b9b7553ecd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.419748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cb07d505-4fcf-4e02-83e3-1b9b7553ecd0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:54.420968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cb07d505-4fcf-4e02-83e3-1b9b7553ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:54.421684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f9e4b-e9d5-4276-a687-069cb0de925c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:54.468838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:54.470270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:54.471191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.471580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.471958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.473362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] RBAC: Adding query filter params (interface=public, service_id=389e627280aa40adb0ba50c59fa08a26, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:54.473848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:54.476670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:54.489080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 379/775] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:44:54 2022] GET /identity/v3/endpoints?service_id=389e627280aa40adb0ba50c59fa08a26&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:54.495799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:54.521341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:54.536840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 397/776] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:44:54.656036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba322-6a1b-482a-ab90-05c04754c54a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:54.702955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:54.703575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.703855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.704132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.704740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:54.706120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:54.718098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 380/777] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:44:54 2022] GET /identity/v3/services => generated 772 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:54.756910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:54.757317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18476cde-8d25-4ee2-802d-def784c582d6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.757599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18476cde-8d25-4ee2-802d-def784c582d6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.758127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18476cde-8d25-4ee2-802d-def784c582d6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.759875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 398/778] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:54.765774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a5547-c638-44f5-af38-2c01d282b0a2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.766347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a5547-c638-44f5-af38-2c01d282b0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.766821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a5547-c638-44f5-af38-2c01d282b0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.786410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5b1a5547-c638-44f5-af38-2c01d282b0a2 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:54.787968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5b1a5547-c638-44f5-af38-2c01d282b0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:54.884915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:54.902332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 399/779] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:44:54.928562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b453e7-3d89-4898-b33f-86fae6c86a37 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:54.974071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:54.974698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:54.974963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:54.975242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] PATH_INFO: `/v3/services/network` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:54.976384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:54.977808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:54.986875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a5547-c638-44f5-af38-2c01d282b0a2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:54.989415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:54.990370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:54.992365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 381/780] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:54 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:54.998944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.106132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a5547-c638-44f5-af38-2c01d282b0a2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.109892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:55.126903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40762875-0359-4761-aad3-8afc81a82750 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.168182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.170535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.171060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.171542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.192289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:55.201002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.201950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.202431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.202906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.203740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] RBAC: Adding query filter params (name=network) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:55.204250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.205852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:55.206655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:55.207552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:44:55.208394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:55.217424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 400/781] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:44:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:55.219585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.223912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 382/782] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:44:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:55.231072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.235265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:55.347125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7dd0a-c400-4dc0-ae7f-bbf8b12fbc4a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.355456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f830be-7b6f-419f-b595-4beaa59abd22 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.383908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.384840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.385288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.385742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.387044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:55.387500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.389048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.394621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.395229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.395529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.396179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.396502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] RBAC: Adding query filter params (type=network) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:55.396804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.398229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.403547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 383/784] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:55.403796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 401/784] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:55.405077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.406579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.436715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:55.484055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:55.516987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae5d56-6e5c-4b19-8d3c-911d3555718d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.522379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65be75-80f6-499b-9131-8c8f6096fb1a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.556016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.557477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.557998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.558521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.559943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.560678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.561081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.561492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.571586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.572553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:55.573458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:55.578631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:55.579613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:55.580460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:55.585573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 384/785] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:55.586662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:55.597016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:55.598064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:55.598954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:55.598954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:55.598954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:55.601997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 402/786] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:55.603114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.704339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:55.705855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1724aaa-62a7-4497-bfcb-9260cd9efb7c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.734300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb986f-78a8-4927-98e6-2afd0fc37476 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.737872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:55.752807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.753991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.754907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.755384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.756281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] RBAC: Adding query filter params (interface=public, service_id=ce15aa152278460cac0fc22058815e41, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:55.756788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.759069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.766946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 385/787] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/endpoints?service_id=ce15aa152278460cac0fc22058815e41&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:55.768014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.782020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.783208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.784012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.784491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.785932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:55.786572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.789385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.806833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 403/788] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:55.806833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.897115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8dbe3-20ff-4a31-902f-86d2bda268a1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.908860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c2078-b850-4e90-bc9d-5426631cad0a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:55.941663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.942279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.942552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.942867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.943510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:55.943807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.944838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:55.945710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:55.945821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.946299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:55.946767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:55.947803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:55.949931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:55.957138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 404/789] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/services => generated 772 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:55.958235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:55.963335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:55.966139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 386/790] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:55.972657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.084424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:56.164665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3ffca-7c46-4862-9058-8f8a0572e4e4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.192860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744a036-c0f0-4587-9b1d-50bc2a1fdc7b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.203796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.204866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.205371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.205872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.214777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.216819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.228798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.229459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.229734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.230001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:56.233921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:56.234809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:44:56.235753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:56.238343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 387/791] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:56.245009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.246604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.249003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:56.262651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:56.263635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:56.264501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:56.264501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:56.264501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:56.265739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 405/792] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:56.269861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.353568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:56.380896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90040d-8a6f-42a8-b0ca-68548da15f34 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.383206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9cb61-115b-41df-9de9-fd992da2e2cc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.400179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:56.445100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.446136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.446993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.447871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.448388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.448906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.449426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.449893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.450815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:56.451665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:56.452184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.452680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.454352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.455933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.484205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 406/793] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:56.488036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 388/794] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:56.489798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.496299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.615359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bae29-1012-4855-b4eb-b471404376a2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.624993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3561f5-a874-4520-b0d7-cc41c2fc60db None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.646628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:56.656860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.657922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.658449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.658993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.667414 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdcda1-6ada-4cee-84c0-ba770018eb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.668049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdcda1-6ada-4cee-84c0-ba770018eb94 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.668324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdcda1-6ada-4cee-84c0-ba770018eb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.668591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdcda1-6ada-4cee-84c0-ba770018eb94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.672249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.673976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.677412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdcda1-6ada-4cee-84c0-ba770018eb94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.678891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdcda1-6ada-4cee-84c0-ba770018eb94 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:56.692035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:56.693031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:56.694237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:56.694237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:56.694237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-81fdcda1-6ada-4cee-84c0-ba770018eb94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:56.694939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:56.695844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 407/795] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594a002-d1b3-4a45-909e-e7559fc3e908 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.696794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594a002-d1b3-4a45-909e-e7559fc3e908 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.697515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594a002-d1b3-4a45-909e-e7559fc3e908 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.697515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 408/796] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:56.699416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 389/797] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:56.701300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594a002-d1b3-4a45-909e-e7559fc3e908 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.704914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.729455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:56.821761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610d179-3fb1-4014-87d8-77fb6e256af9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.849830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594a002-d1b3-4a45-909e-e7559fc3e908 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:56.857958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b88730-749e-43a4-b0e8-8eb4c98611bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.858567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b88730-749e-43a4-b0e8-8eb4c98611bc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.858839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b88730-749e-43a4-b0e8-8eb4c98611bc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.859108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b88730-749e-43a4-b0e8-8eb4c98611bc None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.859745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b88730-749e-43a4-b0e8-8eb4c98611bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:56.860042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b88730-749e-43a4-b0e8-8eb4c98611bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.861492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b88730-749e-43a4-b0e8-8eb4c98611bc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.878726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 390/798] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:56.880008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd94130-6822-4974-a342-46eae75dd684 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.880190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd94130-6822-4974-a342-46eae75dd684 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.880453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd94130-6822-4974-a342-46eae75dd684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.894863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:56.896156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:56.896642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:56.897125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:56.898651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:56.899156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:56.900323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5dd94130-6822-4974-a342-46eae75dd684 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:56.901027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:56.901534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5dd94130-6822-4974-a342-46eae75dd684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:56.921898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 409/799] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:56.922846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:56.964705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:56.990655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 391/800] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:44:56.992375 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd94130-6822-4974-a342-46eae75dd684 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.016715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:57.089287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64907bb2-93a7-43b3-9244-323fe75550ba None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.154240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c892a5-4bed-4f3f-b8a4-26ed071a5f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.154980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c892a5-4bed-4f3f-b8a4-26ed071a5f04 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.155278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c892a5-4bed-4f3f-b8a4-26ed071a5f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.155549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c892a5-4bed-4f3f-b8a4-26ed071a5f04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.162710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c892a5-4bed-4f3f-b8a4-26ed071a5f04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.164293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c892a5-4bed-4f3f-b8a4-26ed071a5f04 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.168192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd94130-6822-4974-a342-46eae75dd684 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c2c892a5-4bed-4f3f-b8a4-26ed071a5f04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:57.176480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:57.177391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:57.178279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 410/801] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:57.179823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f8db-9f76-49c1-830a-17fe05e9bbdf None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.180102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f8db-9f76-49c1-830a-17fe05e9bbdf None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.180368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f8db-9f76-49c1-830a-17fe05e9bbdf None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.181924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 411/802] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:57.188235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24f8db-9f76-49c1-830a-17fe05e9bbdf None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.216616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3e7f8-071e-4224-a1f9-0545184928f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.217184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3e7f8-071e-4224-a1f9-0545184928f5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.217509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3e7f8-071e-4224-a1f9-0545184928f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.217781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3e7f8-071e-4224-a1f9-0545184928f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.229836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3e7f8-071e-4224-a1f9-0545184928f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.231278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3e7f8-071e-4224-a1f9-0545184928f5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-37d3e7f8-071e-4224-a1f9-0545184928f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:57.248046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:57.248973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:57.249871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:57.249871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:57.249871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:57.253966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 392/803] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:57.255757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2414-c434-4895-9513-4d6103f0c17f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.256182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2414-c434-4895-9513-4d6103f0c17f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.256462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2414-c434-4895-9513-4d6103f0c17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.277901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f5ba2414-c434-4895-9513-4d6103f0c17f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:57.279558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f5ba2414-c434-4895-9513-4d6103f0c17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:57.296201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:57.306807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24f8db-9f76-49c1-830a-17fe05e9bbdf None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.343729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.344323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.344598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.344866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.345569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:57.345867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.347295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.349989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:57.369232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 412/804] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:57.369232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.369497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 393/805] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:44:57.369497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba2414-c434-4895-9513-4d6103f0c17f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.487778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba2414-c434-4895-9513-4d6103f0c17f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.492219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba423f-9fbe-4d29-81f4-16734bd567cb None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.527835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.528480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.528757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.529081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.529747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:57.530048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.531416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.534785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.535720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.536194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.536660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.544448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 413/806] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:57.545297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.554607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:44:57.561462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:57.562356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:44:57.563226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:57.565731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:57.571821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 394/807] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:57.571821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.642684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:57.660533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0a232-234d-4a01-9605-9be40f90d1f0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.695831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33368b63-4958-423c-8c4f-c588ecc342c5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.702432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.703489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.703773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.704045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/b3e63489796544afaa44deab302681b9/roles/3a9af0f0d1d04401a0fee6860911b19f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.719343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=b3e63489796544afaa44deab302681b9, role_id=3a9af0f0d1d04401a0fee6860911b19f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.720773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.731029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.731656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.731929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.732196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.734408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 414/808] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:57 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/b3e63489796544afaa44deab302681b9/roles/3a9af0f0d1d04401a0fee6860911b19f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:57.739227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.740636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.746176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:44:57.754453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:57.755455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 395/809] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:57.757082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.842261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:57.880895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed3ca2-8785-45b6-83b2-fb2bd8056938 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.896666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:57.906031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eed878-cc51-4377-9518-655bc84c8d83 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:57.924769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.925448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.925729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.926002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.926592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.927975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:57.938013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:57.939108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:57.940066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 396/810] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:57.941009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:57.954786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:57.955706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:57.956345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:57.956625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:57.957605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:57.957923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:57.959649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:57.974460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 415/811] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:57.975310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.059041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabf077-61c7-4b58-9942-9b759c074c82 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.097292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.098305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.098791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.099262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.099781 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d41af-406a-46a2-a4f2-d9b4ce05d9f9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.100400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:58.101507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.103128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.116029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 397/812] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:58.121976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.127670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:58.139216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.139903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.140497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.140682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.141364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:58.141661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.143173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.153590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 416/813] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 14:44:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:58.153590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.170667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:58.258526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95e9d3-9973-44a2-8a4a-843769425e3f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.295599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8945-ef81-4703-bdb3-f98293212969 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.299442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.299592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.299953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.300319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.312033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.313789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:44:58.331127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:58.332358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:44:58.333513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:44:58.333513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:44:58.333513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:44:58.340169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 398/814] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:58.341074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.347552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.348461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.348894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.349362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/552b3ea5ceab4114908b8108d415c9cd/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.352033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=552b3ea5ceab4114908b8108d415c9cd, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.354115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.377388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 417/815] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:44:58 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/552b3ea5ceab4114908b8108d415c9cd/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:44:58.378596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.457826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:58.489790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:58.506424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa48a3c-882b-42c0-9f8a-a9990a439415 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.521010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2cd4b-0a7b-4b91-bc58-688ecc9f7052 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.554990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83975b9-7c26-4772-b82c-b4724abed65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.555806 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83975b9-7c26-4772-b82c-b4724abed65e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.556199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83975b9-7c26-4772-b82c-b4724abed65e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.556473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83975b9-7c26-4772-b82c-b4724abed65e None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.557103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83975b9-7c26-4772-b82c-b4724abed65e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:58.557456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83975b9-7c26-4772-b82c-b4724abed65e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.559202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83975b9-7c26-4772-b82c-b4724abed65e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.569332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 399/816] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 14:44:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:58.569332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503bd30-fa29-4c55-927c-469f8b437514 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.569332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503bd30-fa29-4c55-927c-469f8b437514 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.569332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503bd30-fa29-4c55-927c-469f8b437514 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.570269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 400/817] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:44:58.572995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9877e-4de9-4e8a-b77f-da2217625f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.573185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503bd30-fa29-4c55-927c-469f8b437514 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.573784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9877e-4de9-4e8a-b77f-da2217625f0d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.574134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9877e-4de9-4e8a-b77f-da2217625f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.574470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9877e-4de9-4e8a-b77f-da2217625f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.575439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9877e-4de9-4e8a-b77f-da2217625f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:58.575816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9877e-4de9-4e8a-b77f-da2217625f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.577293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9877e-4de9-4e8a-b77f-da2217625f0d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.590976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 418/818] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:58.592267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e98f3-a69c-4c4e-a685-49a9d0b926c0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.592550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e98f3-a69c-4c4e-a685-49a9d0b926c0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.592822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e98f3-a69c-4c4e-a685-49a9d0b926c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.615964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-065e98f3-a69c-4c4e-a685-49a9d0b926c0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:44:58.617929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-065e98f3-a69c-4c4e-a685-49a9d0b926c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:44:58.701160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503bd30-fa29-4c55-927c-469f8b437514 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.707382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 419/819] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:44:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:44:58.708359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e98f3-a69c-4c4e-a685-49a9d0b926c0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.753351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:58.786484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.787105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.787384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.787656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.788883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.790277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.815090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e98f3-a69c-4c4e-a685-49a9d0b926c0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:58.855247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:58.856150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:58.856599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:58.856965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:58.860062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 401/820] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:44:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:44:58.865175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:58.866958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:44:58.880185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:58.881310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:44:58.882828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:58.885088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 420/821] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:44:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:58.888551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:58.901790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:59.040708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecf275-474d-44ee-891d-15421839c06a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:59.062318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:59.088637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:59.090532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:59.090812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:59.092457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:59.093225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:59.093528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:59.094976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:59.128091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 402/822] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:44:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:59.134162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:59.218743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:59.222510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65b83-74c8-4cf8-9163-3240a513b54e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:59.297376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:59.298039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:59.298363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:59.298637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] PATH_INFO: `/v3/services/network` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:59.299232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:59.300628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:44:59.317264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:44:59.318308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:44:59.320100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 421/823] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:44:59 2022] GET /identity/v3/services/network => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:44:59.325247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:59.364356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:59.443092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6db0-32fd-4836-9a71-8b4a03e0a811 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:59.510532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:59.511147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:59.511425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:59.511952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:59.526282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dd87c-322e-43ff-92e2-25ff25909e11 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:59.586510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:59.587128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:59.587404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:59.587681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:59.588314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] RBAC: Adding query filter params (name=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:59.588613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:59.590020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:59.600467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 422/824] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:44:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:44:59.606582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:59.608803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:59.610495 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:59.657027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:59.681929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:44:59.724968 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 403/825] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:44:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:44:59.731321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:59.795244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bb5f3-d384-491b-a98b-6a3aa2e02874 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:44:59.861917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:44:59.862545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:44:59.862816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:44:59.863087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:44:59.863723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:44:59.864022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:44:59.866679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:44:59.883095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 423/826] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:44:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:44:59.888372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:44:59.931680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5f57a-a2e8-40fe-a88c-fa9773f8ee30 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.005572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ecb8e-dcb3-4255-b544-0b81ba5ae477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.009019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ecb8e-dcb3-4255-b544-0b81ba5ae477 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.009931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ecb8e-dcb3-4255-b544-0b81ba5ae477 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.010561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ecb8e-dcb3-4255-b544-0b81ba5ae477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.030982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ecb8e-dcb3-4255-b544-0b81ba5ae477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:00.040185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ecb8e-dcb3-4255-b544-0b81ba5ae477 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-701ecb8e-dcb3-4255-b544-0b81ba5ae477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:00.065942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:00.066945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:00.073356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:00.073356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:00.073356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:00.073356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 404/827] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:00.073356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735494-e3f5-4955-b3e1-38f44323545e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.073356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735494-e3f5-4955-b3e1-38f44323545e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.073356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735494-e3f5-4955-b3e1-38f44323545e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.075502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 405/828] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:00.076715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb04c8-14f7-459d-a1fe-e06512d6f76d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.076995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb04c8-14f7-459d-a1fe-e06512d6f76d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.077296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb04c8-14f7-459d-a1fe-e06512d6f76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.082452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:00.098311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18b425-c6ca-4f35-a709-337b8919b5f6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.100082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-84bb04c8-14f7-459d-a1fe-e06512d6f76d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:00.101387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-84bb04c8-14f7-459d-a1fe-e06512d6f76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:00.111309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:00.146030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.147054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.147618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.148095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.149101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:00.150721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:00.179357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 424/829] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:44:59 2022] POST /identity/v3/endpoints => generated 337 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:00.180852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:00.198177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 406/830] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:00.325571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc97d9-9109-40a5-b9c2-697796b26ebc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.368245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.368876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.369159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.369455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.370177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:00.370484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:00.371951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:00.386756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 425/831] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:00.394399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb04c8-14f7-459d-a1fe-e06512d6f76d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:00.526940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb04c8-14f7-459d-a1fe-e06512d6f76d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.562833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.563839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.564435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.564913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.572588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:00.574309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:00.577080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:00.582476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:00.596540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:00.596947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:00.597826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:00.598708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:00.607281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 407/832] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:00.610389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:00.728934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b1e32-94f1-4ffd-9b54-c5f40281cfe4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.729518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59737b80-2209-4859-b3b8-fb8c0aabb26e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.778971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.780068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.780600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.780998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.781282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.781780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.782146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.782486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.782734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:00.783402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:00.785292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:00.795209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:00.799350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:00.800970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:00.802116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 426/833] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:45:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:00.806966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:00.813309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 408/834] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:00.817028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:00.891417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:00.894061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:00.930034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942dba3-8879-4e1e-8fac-556365be624f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.950897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dee14-18d6-4fcd-8bf1-1dba6260e314 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:00.968016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.969278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.970249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.971004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.972144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:00.972672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:00.974646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:00.987624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 427/835] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:45:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:00.991883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bdd54-b68a-402b-9cf0-9cb437c22b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:00.992858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bdd54-b68a-402b-9cf0-9cb437c22b8e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:00.993358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bdd54-b68a-402b-9cf0-9cb437c22b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:00.993468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:00.994014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bdd54-b68a-402b-9cf0-9cb437c22b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:00.994916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bdd54-b68a-402b-9cf0-9cb437c22b8e None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=7b6809bf91d244e4bed5b8a256a3f95a) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:00.995424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bdd54-b68a-402b-9cf0-9cb437c22b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:00.998337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bdd54-b68a-402b-9cf0-9cb437c22b8e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:01.013944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 409/836] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:00 2022] GET /identity/v3/role_assignments?user.id=7b6809bf91d244e4bed5b8a256a3f95a&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:01.126505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33279a-f0c2-49df-977b-3d6df88a5b57 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:01.154002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a498-2ceb-489e-9a8e-bc7555e1372c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.154273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a498-2ceb-489e-9a8e-bc7555e1372c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.154510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a498-2ceb-489e-9a8e-bc7555e1372c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.156138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 410/837] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:01.161965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf13e-1338-4631-913b-8b2120122d37 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.162256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf13e-1338-4631-913b-8b2120122d37 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.162519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf13e-1338-4631-913b-8b2120122d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.163991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:01.171912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:01.173906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.174380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.174868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.185304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-11ccf13e-1338-4631-913b-8b2120122d37 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:01.186531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-11ccf13e-1338-4631-913b-8b2120122d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:01.190653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:01.192398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:01.212761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:01.213989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:01.214876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:01.214876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:01.214876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:01.214876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 428/838] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:01.222147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:01.276941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 411/839] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:01.346326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a65e-846a-4766-b686-5dd1cd62259d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:01.384578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78a2a6-9046-4c09-870d-2193d6116e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:01.385112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a2a6-9046-4c09-870d-2193d6116e16 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.385496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a2a6-9046-4c09-870d-2193d6116e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.385770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a2a6-9046-4c09-870d-2193d6116e16 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.386413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78a2a6-9046-4c09-870d-2193d6116e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:01.386966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78a2a6-9046-4c09-870d-2193d6116e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:01.388400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78a2a6-9046-4c09-870d-2193d6116e16 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:01.400989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 429/840] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:01.406478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf13e-1338-4631-913b-8b2120122d37 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:01.452896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44620-b7f0-4aa3-a1c7-cba7b7ec5c7d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.453238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44620-b7f0-4aa3-a1c7-cba7b7ec5c7d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.453475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44620-b7f0-4aa3-a1c7-cba7b7ec5c7d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.455056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 430/841] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:01.459896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc8b30-2564-44c6-a006-d1e5c99a8e11 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.460185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc8b30-2564-44c6-a006-d1e5c99a8e11 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.460464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc8b30-2564-44c6-a006-d1e5c99a8e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.467339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:01.481743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b0dc8b30-2564-44c6-a006-d1e5c99a8e11 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:01.483237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b0dc8b30-2564-44c6-a006-d1e5c99a8e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:01.512377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf13e-1338-4631-913b-8b2120122d37 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:01.522840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:01.548070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:01.548861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.549144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.549444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.557874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:01.559304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:01.565523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 431/842] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:01.570280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc8b30-2564-44c6-a006-d1e5c99a8e11 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:01.571408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:01.572382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:01.573342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 412/843] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:45:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:01.577992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:01.677307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc8b30-2564-44c6-a006-d1e5c99a8e11 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:01.693178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef83c67-e106-4427-bddb-16e0baf36639 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:01.708031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:01.712902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:01.713515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.713794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.714110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.724622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:01.730114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:01.731262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:01.732508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 432/844] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:45:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:01.736814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:01.738646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:01.739260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.739539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.739809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.740532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:01.740834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:01.742422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:01.761155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 413/845] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 14:45:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:01.766387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:01.771064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:01.856321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490b0c4-50e8-41b3-9ef3-28e90cfb641e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:01.893465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:01.894068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.894345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.894615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.895272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:01.895576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:01.895739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ce949-bbc7-4e49-b6ad-719dec4f3836 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:01.896935 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:01.911120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 433/846] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:45:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:01.911879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:01.932261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:01.932813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:01.933095 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:01.933420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:01.977759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.007373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.008244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.008812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.024348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3577ab-b59e-43d4-a7ef-6004bf98607f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.062546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.063181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.063459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.063729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.064308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.065721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:02.075097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:02.076063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:02.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 434/847] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:02.077713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.127504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 414/848] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:45:02.128337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.191546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfaba1-32b9-46f6-8ef7-e0193c0d1fa2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.198324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.228600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.228889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3c6dd-5e5c-4933-a198-b7741e02721f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.229248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.229531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.229803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.239991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.242006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:02.257301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:02.258396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:02.259263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:02.259263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:02.259263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:02.259263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 435/849] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:02.259263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.271965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.272581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.272859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.272961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.273127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.273856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:02.274094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.275446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.284351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 415/850] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:02.284351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.368739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355783e-3bde-4335-8402-d56928b7de2f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.385354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9930a9-2295-4196-859e-6f3b260f40ce None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.404348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.404974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.405281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.405554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.415827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.417291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.420781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.421417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.421694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.421964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.422598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:02.422899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.424237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.426595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:02.432465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:02.433621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:02.434709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:02.434709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:02.434709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:02.436177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 436/851] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:02.437091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.443386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 416/852] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:02.445007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.503734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.546863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd64b04-2b38-4571-a5aa-533fdb6fb11d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.551496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b909d-5345-4f46-975e-c97223b42c87 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.579956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.580549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.580829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.581099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.581754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:02.582052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.583410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.587973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 437/853] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:02.588411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.588757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.589425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.589704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.589975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.590619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:02.590920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.592293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.613247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 417/854] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:02.615484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.682928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.733929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9549e-2050-413e-a2ed-c929d33499ee None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.739933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622b94-f5c8-4b33-9b84-f12d583b53ff None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.781314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.782828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.783617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.783966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.784337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.796369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.796799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.797917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.797917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.798082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.798155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.809956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:02.810214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:02.811037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:02.811918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:02.812349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:02.812421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 418/855] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:02.818452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:02.824387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:02.825564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 14:45:02.826716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:02.830303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 438/856] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:02.831386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:02.944119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0b7ca-9327-43ae-b5b9-73a266eac970 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.961268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:02.968707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafa933-e3e3-4736-854e-bf9c42b03727 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:02.981899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:02.982669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:02.983029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:02.983391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:02.984235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:02.984577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:02.986169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:03.004415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 419/857] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:03.005461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:03.033239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:03.034322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.034853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.035682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.036856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:03.037351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:03.040373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:03.064186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 439/858] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 14:45:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:03.069782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:03.120516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d891e40-3ffc-4f78-96c7-245d6c89ade9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:03.125970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:03.158470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c743605-0de0-492f-ab20-5689ac6706b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:03.159229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c743605-0de0-492f-ab20-5689ac6706b6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.159596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c743605-0de0-492f-ab20-5689ac6706b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.159963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c743605-0de0-492f-ab20-5689ac6706b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.160774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c743605-0de0-492f-ab20-5689ac6706b6 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=b3e63489796544afaa44deab302681b9) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:03.161783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c743605-0de0-492f-ab20-5689ac6706b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:03.163569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c743605-0de0-492f-ab20-5689ac6706b6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:03.177223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 420/859] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:03 2022] GET /identity/v3/role_assignments?user.id=b3e63489796544afaa44deab302681b9&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:03.203453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee48fd7-ff4a-49ac-9fe0-1aa56ab85bdc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:03.246842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:03.247428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.247706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.247973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.331401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:03.333374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:03.345080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8106236-05b8-46b4-8004-be00e1e6f10e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.345416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8106236-05b8-46b4-8004-be00e1e6f10e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.345672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8106236-05b8-46b4-8004-be00e1e6f10e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.347285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 421/860] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:03.353636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352e928-2cd5-4e74-8168-d6a6b7c04cdc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.353963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352e928-2cd5-4e74-8168-d6a6b7c04cdc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.354199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352e928-2cd5-4e74-8168-d6a6b7c04cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.375276 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7352e928-2cd5-4e74-8168-d6a6b7c04cdc None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:03.376997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7352e928-2cd5-4e74-8168-d6a6b7c04cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:03.393708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:03.427766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:03.473244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 440/861] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:45:03.474812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 422/862] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:03.475977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:03.580989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10990dce-8943-4c19-9d3f-3e799ccf938b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:03.618698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:03.619333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.619616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.619892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.630049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:03.631501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:03.634906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:03.648513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:03.649575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:03.650456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:03.650456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:03.650456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:03.653152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 441/863] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:03.658945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352e928-2cd5-4e74-8168-d6a6b7c04cdc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:03.756073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:03.761471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352e928-2cd5-4e74-8168-d6a6b7c04cdc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:03.799831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:03.800435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.800708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.800973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.801640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:03.801934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:03.803921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:03.825296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 423/864] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:03.827639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:03.863513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:03.878281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a12487-2188-46e8-a66d-07ee8b9dd0f0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:03.922793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:03.923430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.923707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.923975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:03.924931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:03.926676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:03.937988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:03.938976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:03.939902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 442/865] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 9 14:45:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:03.945292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:03.955966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94486a-d3de-45ec-b599-c144d7c6a322 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:03.991721 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:03.992641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:03.993099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:03.993599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:04.000900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:04.002646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:04.015370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:04.016322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:04.017254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:04.019650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 424/866] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:04.021778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:04.028542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:04.081644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3753d7-8bfa-454d-ba38-014c17b946a7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:04.127278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:04.128356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:04.128890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:04.129387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:04.130461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:04.130967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:04.133187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:04.139464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 443/867] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:04.145637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:04.151253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:04.158087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d3687-b93b-4d73-95fb-1f48c88b6d2e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:04.201294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:04.201910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:04.202186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:04.202454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:04.203133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:04.203432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:04.205335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:04.218540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 425/868] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:04.229268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:04.276103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd40098-e80d-4922-81b5-bc5f51c398bf None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:04.301312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:04.315270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2704a-fac0-43ad-805e-44a99bf83a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:04.315927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2704a-fac0-43ad-805e-44a99bf83a53 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:04.316207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2704a-fac0-43ad-805e-44a99bf83a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:04.317365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2704a-fac0-43ad-805e-44a99bf83a53 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:04.318033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2704a-fac0-43ad-805e-44a99bf83a53 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:04.318343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2704a-fac0-43ad-805e-44a99bf83a53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:04.319966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2704a-fac0-43ad-805e-44a99bf83a53 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:04.324921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 444/869] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:04.356333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aed1cb-d472-4cb2-9e94-8de9efc07f4b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:04.396859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b04e9-1bea-49e7-b432-c469bfd98fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:04.397487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b04e9-1bea-49e7-b432-c469bfd98fde None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:04.397769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b04e9-1bea-49e7-b432-c469bfd98fde None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:04.398044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b04e9-1bea-49e7-b432-c469bfd98fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:04.400067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b04e9-1bea-49e7-b432-c469bfd98fde None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=552b3ea5ceab4114908b8108d415c9cd) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:04.400381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b04e9-1bea-49e7-b432-c469bfd98fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:04.401963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b04e9-1bea-49e7-b432-c469bfd98fde None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:04.409328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 426/870] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:04 2022] GET /identity/v3/role_assignments?user.id=552b3ea5ceab4114908b8108d415c9cd&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:04.858135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a027cc-e4a0-4d11-82f7-aba2fb6f099c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:04.858478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a027cc-e4a0-4d11-82f7-aba2fb6f099c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:04.858582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a027cc-e4a0-4d11-82f7-aba2fb6f099c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:04.860157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 445/871] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:04.865445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8484-6a7c-40cf-9ab7-97e6515eec53 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:04.865910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8484-6a7c-40cf-9ab7-97e6515eec53 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:04.866014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8484-6a7c-40cf-9ab7-97e6515eec53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:04.885020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3f5f8484-6a7c-40cf-9ab7-97e6515eec53 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:04.886235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3f5f8484-6a7c-40cf-9ab7-97e6515eec53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:04.896519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:04.964181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 427/872] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:05.246220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a027cc-e4a0-4d11-82f7-aba2fb6f099c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:05.357513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a027cc-e4a0-4d11-82f7-aba2fb6f099c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:05.395995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66b1d6-25f5-4a5e-8b3e-b530726a0308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:05.396616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66b1d6-25f5-4a5e-8b3e-b530726a0308 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:05.396897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66b1d6-25f5-4a5e-8b3e-b530726a0308 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:05.397240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66b1d6-25f5-4a5e-8b3e-b530726a0308 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:05.397922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66b1d6-25f5-4a5e-8b3e-b530726a0308 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:05.399296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66b1d6-25f5-4a5e-8b3e-b530726a0308 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:05.418261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 446/873] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:45:05 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:05.608943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b0b39-f637-46ce-bd3a-d2801fb53929 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:05.609259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b0b39-f637-46ce-bd3a-d2801fb53929 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:05.609458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b0b39-f637-46ce-bd3a-d2801fb53929 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:05.610935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 428/874] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:05.616653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28105b6e-3870-4e92-937c-54a433aa2156 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:05.616942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28105b6e-3870-4e92-937c-54a433aa2156 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:05.617316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28105b6e-3870-4e92-937c-54a433aa2156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:05.636888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-28105b6e-3870-4e92-937c-54a433aa2156 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:05.638201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-28105b6e-3870-4e92-937c-54a433aa2156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:05.685148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:05.717227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 447/875] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:05.990218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b0b39-f637-46ce-bd3a-d2801fb53929 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:06.108990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b0b39-f637-46ce-bd3a-d2801fb53929 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:06.144988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:06.146064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.146675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.147665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.159204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:06.165060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:06.165993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:06.166810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:06.169444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 429/876] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 14:45:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:06.181254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28105b6e-3870-4e92-937c-54a433aa2156 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:06.280059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28105b6e-3870-4e92-937c-54a433aa2156 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:06.315691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:06.316303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.316584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.316860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.317563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:06.317864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:06.319274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:06.327971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 448/877] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 14:45:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:06.333672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:06.339854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:06.448378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cea56-1bf0-4eef-a87f-d967c1d92fed None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:06.484743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f3935-1109-4e47-89ff-43eedf5a10c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:06.485400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3935-1109-4e47-89ff-43eedf5a10c0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.485680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3935-1109-4e47-89ff-43eedf5a10c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.485949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3935-1109-4e47-89ff-43eedf5a10c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.496320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f3935-1109-4e47-89ff-43eedf5a10c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:06.497785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f3935-1109-4e47-89ff-43eedf5a10c0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-bb6f3935-1109-4e47-89ff-43eedf5a10c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:06.513461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:06.514434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:06.515345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:06.515345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:06.515345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:06.515345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 430/878] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:06.520549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:06.534635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72173c-f651-49b1-b983-4e364b34ae05 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.535299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72173c-f651-49b1-b983-4e364b34ae05 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.535577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72173c-f651-49b1-b983-4e364b34ae05 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.537579 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 431/879] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:06.542621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa104a22-0544-4f42-bdff-b303e67b1826 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.542968 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa104a22-0544-4f42-bdff-b303e67b1826 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.543247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa104a22-0544-4f42-bdff-b303e67b1826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.561244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-fa104a22-0544-4f42-bdff-b303e67b1826 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:06.562426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fa104a22-0544-4f42-bdff-b303e67b1826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:06.563325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:06.624534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:06.643248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54ec0e-d8e9-4e10-a077-886e00549fd8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:06.646201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 432/880] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:06.679772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27473a20-81c9-4cef-a0a0-1f76adc96f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:06.680373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27473a20-81c9-4cef-a0a0-1f76adc96f45 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.680648 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27473a20-81c9-4cef-a0a0-1f76adc96f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.680916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27473a20-81c9-4cef-a0a0-1f76adc96f45 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.681580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27473a20-81c9-4cef-a0a0-1f76adc96f45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:06.681877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27473a20-81c9-4cef-a0a0-1f76adc96f45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:06.683238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27473a20-81c9-4cef-a0a0-1f76adc96f45 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:06.695003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 449/881] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:06.700310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa104a22-0544-4f42-bdff-b303e67b1826 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:06.729866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4145326-93d4-46dd-83c7-d50de1c8f0c7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.730193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4145326-93d4-46dd-83c7-d50de1c8f0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.730473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4145326-93d4-46dd-83c7-d50de1c8f0c7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.732217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 450/882] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:06.738126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b6679-69c8-43f1-af8f-4542f1aa097a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.738399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b6679-69c8-43f1-af8f-4542f1aa097a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.738666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b6679-69c8-43f1-af8f-4542f1aa097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.759007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5a6b6679-69c8-43f1-af8f-4542f1aa097a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:06.760275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5a6b6679-69c8-43f1-af8f-4542f1aa097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:06.813372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa104a22-0544-4f42-bdff-b303e67b1826 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:06.845342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 451/883] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:06.852396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:06.853032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:06.853343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:06.853618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:06.860890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:06.862357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:06.874030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:06.875013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:06.876004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 433/884] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:45:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:06.881180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b6679-69c8-43f1-af8f-4542f1aa097a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:06.909156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:06.919946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:06.976640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:06.989521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b6679-69c8-43f1-af8f-4542f1aa097a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.026366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.026962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.027451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.027874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.028848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:07.029380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.031697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.042456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb9177-d25c-4074-a208-c785c5e7c201 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.051248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 452/885] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 14:45:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:07.057288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:07.087765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.088567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.088895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.089174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.089950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.091497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.114322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 434/886] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:45:06 2022] POST /identity/v3/services => generated 269 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:07.124399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:07.162164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:07.182633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327924-034b-4d13-bc05-52749e616170 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.235532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.236791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.237263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.238196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.241916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf73f4-0493-4a40-8993-8a7f376f17a4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.251775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:07.285974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.286767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.287268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.287505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.288797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.290240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:07.302923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:07.304781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:07.305864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:07.307876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 435/887] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:07.313724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:07.335862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.337916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.435997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:07.437792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b58e5-53cd-4f4b-b7a2-b7a724844cc7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.465284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 453/888] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:07 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 14:45:07.470776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:07.488800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.489457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.489779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.490003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.490645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:07.491485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.492861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.505170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 436/889] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:07.508811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:07.555961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:07.622640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43656b9-73b0-44f3-8e5f-92f354fcbaa6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.634932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6a4f4-eb56-4b0a-8afa-bd2fab6f5a94 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.673514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102dad1-5aff-467a-91bc-ce190b3987fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.673766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.674293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.674574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.674846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.675314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102dad1-5aff-467a-91bc-ce190b3987fd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.675487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:07.675772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.676170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102dad1-5aff-467a-91bc-ce190b3987fd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.676644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102dad1-5aff-467a-91bc-ce190b3987fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.677133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.688440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 437/890] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:07.693097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102dad1-5aff-467a-91bc-ce190b3987fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.695203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102dad1-5aff-467a-91bc-ce190b3987fd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d102dad1-5aff-467a-91bc-ce190b3987fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:07.712953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:07.714023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:07.714928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:07.714928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:07.714928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:07.714928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 454/891] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:07.721358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:07.734800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137724b-a6db-4f6b-93bf-cbc736717574 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.735114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137724b-a6db-4f6b-93bf-cbc736717574 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.735350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137724b-a6db-4f6b-93bf-cbc736717574 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.736991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 455/892] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:07.743032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c438fc-f200-4cb8-8381-543747c124dd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.743308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c438fc-f200-4cb8-8381-543747c124dd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.743577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c438fc-f200-4cb8-8381-543747c124dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.764041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c3c438fc-f200-4cb8-8381-543747c124dd None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:07.765371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c3c438fc-f200-4cb8-8381-543747c124dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:07.800087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:07.846863 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:07.851136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0c4ea-a53d-4b29-a43a-642e18fc7b8a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:07.865608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 456/893] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:07.895075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:07.895970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:07.896254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:07.896529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:07.897175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:07.897555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:07.898917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:07.911488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 438/894] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:07.918210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c438fc-f200-4cb8-8381-543747c124dd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.025006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c438fc-f200-4cb8-8381-543747c124dd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.059543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:08.060375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:08.060655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:08.060925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:08.068257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:08.070391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:08.083179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:08.084186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:08.085689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:08.089119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 457/895] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:08.091112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.152163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.179765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:08.205602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199ec1b-0dec-42af-8f56-c7bc720c493d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.250213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:08.251041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:08.251675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:08.252047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:08.253340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:08.253650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:08.255356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:08.273283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 439/896] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:08.276695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.278295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d4061-18e5-4b61-bd69-73b0d35bf961 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.298240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:08.316890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:08.317668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:08.317952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:08.318320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:08.319029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:08.320957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:08.330729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:08.331705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:08.332526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 458/897] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:08.342102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.393720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64456-67b8-41ab-9bdc-a8b07e099eec None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.435979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:08.436844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:08.437296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:08.437728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:08.438610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] RBAC: Adding query filter params (role.id=e5c78c8957b941eaba90da16737d4525, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=b3e63489796544afaa44deab302681b9) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:08.438916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:08.440351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:08.451203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 440/898] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:08 2022] GET /identity/v3/role_assignments?user.id=b3e63489796544afaa44deab302681b9&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=e5c78c8957b941eaba90da16737d4525 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:08.456186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:08.476625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cac95-251c-4b97-90af-3b508d34a933 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.516008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:08.517065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:08.517595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:08.518085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:08.518974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:08.519492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:08.521111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:08.528117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 459/899] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:08.534276 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.627181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:08.647837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f3868-5f88-4972-86c4-f2ae4c39e542 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.682260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:08.683280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:08.683791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:08.684273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:08.685189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:08.685740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:08.687418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:08.693790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 441/900] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:08.700594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.805030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ef18-cdbc-439f-8725-c5217272aef3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.844419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686be51e-ec0c-40bc-89b3-7a94a83b40b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:08.845030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686be51e-ec0c-40bc-89b3-7a94a83b40b7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:08.845345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686be51e-ec0c-40bc-89b3-7a94a83b40b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:08.845620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686be51e-ec0c-40bc-89b3-7a94a83b40b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:08.846314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686be51e-ec0c-40bc-89b3-7a94a83b40b7 None admin] RBAC: Adding query filter params (interface=public, service_id=7502594797d549e2af7e031af6bacb72, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:08.846618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686be51e-ec0c-40bc-89b3-7a94a83b40b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:08.847984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686be51e-ec0c-40bc-89b3-7a94a83b40b7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:08.855626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 460/901] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:45:08 2022] GET /identity/v3/endpoints?service_id=7502594797d549e2af7e031af6bacb72&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:08.861187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:08.964641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45219d-c2bc-49c2-ba7c-3b53065e58e3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:08.999402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594d99e-f694-4059-9ed8-bbc51f8f20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:09.000427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d99e-f694-4059-9ed8-bbc51f8f20c1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:09.000917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d99e-f694-4059-9ed8-bbc51f8f20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:09.001437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d99e-f694-4059-9ed8-bbc51f8f20c1 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:09.002233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594d99e-f694-4059-9ed8-bbc51f8f20c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:09.003821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594d99e-f694-4059-9ed8-bbc51f8f20c1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:09.023492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 442/902] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:45:08 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:45:09.370286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b45a8-36a7-438a-93a3-9f1790b9da95 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:09.370788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b45a8-36a7-438a-93a3-9f1790b9da95 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:09.370788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b45a8-36a7-438a-93a3-9f1790b9da95 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:09.372310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 461/903] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:09.377872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63234c9-3258-4113-9eae-42da17c12ea7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:09.378289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63234c9-3258-4113-9eae-42da17c12ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:09.378460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63234c9-3258-4113-9eae-42da17c12ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:09.397374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b63234c9-3258-4113-9eae-42da17c12ea7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:09.398551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b63234c9-3258-4113-9eae-42da17c12ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:09.401547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:09.473323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 443/904] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:09.745449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b45a8-36a7-438a-93a3-9f1790b9da95 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:09.762077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:09.851065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b45a8-36a7-438a-93a3-9f1790b9da95 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:09.885228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca955568-dfc4-4e0b-b0e1-ca534c4cc321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:09.885937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca955568-dfc4-4e0b-b0e1-ca534c4cc321 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:09.886210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca955568-dfc4-4e0b-b0e1-ca534c4cc321 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:09.886477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca955568-dfc4-4e0b-b0e1-ca534c4cc321 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:09.887305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca955568-dfc4-4e0b-b0e1-ca534c4cc321 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:09.888666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca955568-dfc4-4e0b-b0e1-ca534c4cc321 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ca955568-dfc4-4e0b-b0e1-ca534c4cc321 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:09.898324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:09.899246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:09.900133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:09.903595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 462/905] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 9 14:45:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:09.906113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63234c9-3258-4113-9eae-42da17c12ea7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:10.022528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63234c9-3258-4113-9eae-42da17c12ea7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:10.039889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb811-6dd3-4150-b1dd-d1a2d192d3d5 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.040208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb811-6dd3-4150-b1dd-d1a2d192d3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.040439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb811-6dd3-4150-b1dd-d1a2d192d3d5 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.042050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 463/906] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:10.047050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a01997-00de-4b71-89d0-cb93e3d85928 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.047357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a01997-00de-4b71-89d0-cb93e3d85928 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.047629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a01997-00de-4b71-89d0-cb93e3d85928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.063985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:10.064921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.065444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.065917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.066848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:10.067090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-86a01997-00de-4b71-89d0-cb93e3d85928 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:10.067556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:10.068284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-86a01997-00de-4b71-89d0-cb93e3d85928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:10.069936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:10.075210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 444/907] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:10.082661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:10.085987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:10.147916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 464/908] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:10.190246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06287d-7893-42d7-8233-c3fe285ed245 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:10.224856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:10.226012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.226285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.226548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.227161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:10.227453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:10.228797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:10.238553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 445/909] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:10.239996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a01997-00de-4b71-89d0-cb93e3d85928 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:10.252258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:10.348506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a01997-00de-4b71-89d0-cb93e3d85928 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:10.384030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:10.384635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.384890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.385158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.385873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] RBAC: Adding query filter params (interface=public, service_id=88b9a9d295c94f60b108c38f1d3c9775, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:10.386171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:10.387780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:10.397309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 465/910] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:45:10 2022] GET /identity/v3/endpoints?service_id=88b9a9d295c94f60b108c38f1d3c9775&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:10.398975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:10.439120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:10.472924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:10.514462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7ca7b-0684-4cbf-82f3-960a07ab2825 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:10.531619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:10.556196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445d09f-b839-4372-b272-669fd69c41e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:10.556820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d09f-b839-4372-b272-669fd69c41e0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.557092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d09f-b839-4372-b272-669fd69c41e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.557408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d09f-b839-4372-b272-669fd69c41e0 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.558002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445d09f-b839-4372-b272-669fd69c41e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:10.559410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445d09f-b839-4372-b272-669fd69c41e0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:10.573262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 446/911] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:45:10 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:45:10.574921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df701126-84a2-406e-89b4-e7f1773b6ba0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:10.617465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81e6ee-f6ae-40c9-8a34-f7839bc2c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:10.618006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81e6ee-f6ae-40c9-8a34-f7839bc2c5af None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.618281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81e6ee-f6ae-40c9-8a34-f7839bc2c5af None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.618550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81e6ee-f6ae-40c9-8a34-f7839bc2c5af None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.619206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81e6ee-f6ae-40c9-8a34-f7839bc2c5af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:10.620585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81e6ee-f6ae-40c9-8a34-f7839bc2c5af None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:10.640022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 466/912] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:45:10 2022] POST /identity/v3/services => generated 241 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:10.691907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6938f-b36b-420f-ba7d-431928b7658c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.692299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6938f-b36b-420f-ba7d-431928b7658c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.692468 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6938f-b36b-420f-ba7d-431928b7658c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.694247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 447/913] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:10.700118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2eec-824b-4606-9e03-55ef17e22591 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:10.700441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2eec-824b-4606-9e03-55ef17e22591 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:10.700659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2eec-824b-4606-9e03-55ef17e22591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:10.722571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-924f2eec-824b-4606-9e03-55ef17e22591 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:10.723806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-924f2eec-824b-4606-9e03-55ef17e22591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:10.810622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 467/914] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:11.077734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6938f-b36b-420f-ba7d-431928b7658c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:11.190534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6938f-b36b-420f-ba7d-431928b7658c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:11.223247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:11.228429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daa5db-aea9-4ca1-bdcd-e0a917ffd71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:11.229137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daa5db-aea9-4ca1-bdcd-e0a917ffd71d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.229436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daa5db-aea9-4ca1-bdcd-e0a917ffd71d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.229740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daa5db-aea9-4ca1-bdcd-e0a917ffd71d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.240645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8daa5db-aea9-4ca1-bdcd-e0a917ffd71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e8daa5db-aea9-4ca1-bdcd-e0a917ffd71d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:11.245930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:11.246855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:11.248036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:11.251978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 448/915] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:11.253373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2eec-824b-4606-9e03-55ef17e22591 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:11.294313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a2b27-428a-459c-ae65-9b9fd30e3aae None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.294502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a2b27-428a-459c-ae65-9b9fd30e3aae None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.297156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a2b27-428a-459c-ae65-9b9fd30e3aae None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.297156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 449/916] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:11.302432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878407b8-65c0-406d-8bed-d89d7ab6b59e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.302706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878407b8-65c0-406d-8bed-d89d7ab6b59e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.302977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878407b8-65c0-406d-8bed-d89d7ab6b59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.317876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:11.327111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-878407b8-65c0-406d-8bed-d89d7ab6b59e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:11.328473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-878407b8-65c0-406d-8bed-d89d7ab6b59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:11.368593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2eec-824b-4606-9e03-55ef17e22591 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:11.409681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:11.411254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.411673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.412232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.413396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:11.414110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:11.415740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:11.423201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 450/917] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:11.431320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 468/918] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:11.436812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878407b8-65c0-406d-8bed-d89d7ab6b59e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:11.540439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878407b8-65c0-406d-8bed-d89d7ab6b59e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:11.563340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:11.575486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:11.576119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.576393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.576659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.587516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:11.588991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:11.605573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:11.606594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:11.607458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:11.607458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:11.607458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:11.610639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 451/919] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:11.612806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:11.716209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25a0b-04c9-4b88-b747-f1a2dc7922cc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:11.718414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:11.739925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:11.755990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:11.756662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.756926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.757221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.757862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:11.758162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:11.759618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:11.772626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 469/920] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:11.778528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:11.851156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085792e8-8d44-4c80-9740-874501851d04 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:11.897913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:11.898786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.899163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.899590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.900363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:11.901928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:11.909300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b42f5-d681-40d2-b713-12064bad05df None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:11.914130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:11.915261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 14:45:11.916092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:11.921282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 452/921] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:11.922441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:11.951368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:11.952055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:11.952341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:11.952615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:11.960743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:11.962508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:11.971381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:11.976498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:11.977498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:11.978456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:11.981987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 470/922] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:11.987162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.034433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:12.052364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89c5f8-223b-4948-a880-e96c8458a072 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.091844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.092453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.092730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.093000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.093906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:12.094321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.095686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.100936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e001dd-b024-45d0-ba0f-9983705e907e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.106661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 453/923] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:12.107549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.137597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.138602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.139180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.139651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.140586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:12.141154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.143014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.162052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 471/924] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:12.172525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.221966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d71c18-9b33-4ee2-94d5-335636a902fb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.223549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:12.262258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.263583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.264281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.264701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.265818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:12.266658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.268185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.281247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 454/925] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 14:45:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:12.283764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.294687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:12.298849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac0660-059c-4d75-8d5b-c98742f32dc6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.339346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.340337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.340818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.341362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.356752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.359008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:12.379570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:12.380553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:12.382000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:12.382000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:12.382000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:12.385613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 472/926] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:12.391385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.412037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dee9b-20fc-45ae-98fb-c2fae0fbf4f9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.448216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.448846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.449106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.449420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.450084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.451712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.484078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 455/927] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:45:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:12.530254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b363ed-bf96-413c-b517-721273e38410 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.575184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.575797 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.576073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.576339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.576973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:12.577301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.578813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.597382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 473/928] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:12.599692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.633600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:12.711464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b408dda-84de-424e-bee3-0a4bb8559366 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.748454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048e00c-028c-463a-acf9-ca0063cd36ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.749178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048e00c-028c-463a-acf9-ca0063cd36ed None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.749481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048e00c-028c-463a-acf9-ca0063cd36ed None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.749748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048e00c-028c-463a-acf9-ca0063cd36ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.758362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048e00c-028c-463a-acf9-ca0063cd36ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.759913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048e00c-028c-463a-acf9-ca0063cd36ed None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e048e00c-028c-463a-acf9-ca0063cd36ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:12.773368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:12.774358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:12.775228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:12.779647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 456/929] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:12.781137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.790912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:12.865025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6f89c-0d36-4618-9df8-16ce7fa948c3 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.865395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6f89c-0d36-4618-9df8-16ce7fa948c3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.866128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6f89c-0d36-4618-9df8-16ce7fa948c3 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.867736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 457/930] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:12.872744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ade6-2a99-4322-951d-fba784491b7f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.873026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ade6-2a99-4322-951d-fba784491b7f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.873340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ade6-2a99-4322-951d-fba784491b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.895301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ea81ade6-2a99-4322-951d-fba784491b7f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:12.896098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8b5d0-a757-443d-abb5-715d2ceba1f7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:12.896406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ea81ade6-2a99-4322-951d-fba784491b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:12.930997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e307cb8-769c-448b-b29a-538975911db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:12.931612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e307cb8-769c-448b-b29a-538975911db9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.931893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e307cb8-769c-448b-b29a-538975911db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.932171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e307cb8-769c-448b-b29a-538975911db9 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.932852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e307cb8-769c-448b-b29a-538975911db9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:12.933157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e307cb8-769c-448b-b29a-538975911db9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:12.934719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e307cb8-769c-448b-b29a-538975911db9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:12.953270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 474/931] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:12.953270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751db35-10a1-43cf-b168-df470db119fd None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.953270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751db35-10a1-43cf-b168-df470db119fd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.953270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751db35-10a1-43cf-b168-df470db119fd None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.953270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 475/932] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:12.955216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751db35-10a1-43cf-b168-df470db119fd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:12.972881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 458/933] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:12.972881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb380ab-45d2-4a52-b2fb-d4d25eaf3de0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:12.972881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb380ab-45d2-4a52-b2fb-d4d25eaf3de0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:12.972881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb380ab-45d2-4a52-b2fb-d4d25eaf3de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:12.996247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5fb380ab-45d2-4a52-b2fb-d4d25eaf3de0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:12.997561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5fb380ab-45d2-4a52-b2fb-d4d25eaf3de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:13.009322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:13.046351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:13.071166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751db35-10a1-43cf-b168-df470db119fd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.088777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 459/934] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:13.107242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.107990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.108341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.108711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/b3e63489796544afaa44deab302681b9/roles/e5c78c8957b941eaba90da16737d4525` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.111001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=b3e63489796544afaa44deab302681b9, role_id=e5c78c8957b941eaba90da16737d4525)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.112514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.129387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 476/935] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:45:12 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/b3e63489796544afaa44deab302681b9/roles/e5c78c8957b941eaba90da16737d4525 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:45:13.259157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb380ab-45d2-4a52-b2fb-d4d25eaf3de0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.363966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb380ab-45d2-4a52-b2fb-d4d25eaf3de0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.398812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.399810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.400320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.400794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.401622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.401952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.403546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:13.418695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:13.419687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 14:45:13.420578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:13.423432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 460/936] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 9 14:45:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:13.429996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.440355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:13.521089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea60b-56b2-4ae0-9214-d0670fc78d1f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.550489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:13.554456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bd5c5-c450-49b2-ae47-e2a1db5133d1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.563501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.564513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.565003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.565516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.566408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.567994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:13.577814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:13.578804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:13.579628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:13.582719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 477/937] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:13.594750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.595414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.596249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.596749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.597349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.598216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:13.598788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.601083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.611260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 461/938] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:13.612470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.692447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:13.719445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0bbcc-f4af-449f-b0fa-dbf91d491ffd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.728491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0957fd-ab1f-4653-acb8-46a7b8661885 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.755388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.755993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.756317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.756665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.757318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:13.757619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.759158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.764630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 478/939] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:13.766955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.767797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.768293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.768792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.769733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:13.770060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.771465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.774238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.781790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 462/940] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:13.782707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.848769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:13.889390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e91ca-0511-4c9c-9661-0ad518632872 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.908898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f746a5-cff3-4b6b-b347-d1319f5b42f1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:13.926045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.926918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.927427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.927903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.928806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:13.929413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.931039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.938640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 479/941] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:13.944975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:13.956805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:13.957876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:13.957876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:13.958282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:13.959050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:13.961210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:13.966603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:13.993831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 463/942] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:45:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:14.069403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd805904-640a-49cf-ac2b-7489ecc188d0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:14.090537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:14.115984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90780ffd-b5ea-461d-99c1-8e6d2fe3fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:14.116892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780ffd-b5ea-461d-99c1-8e6d2fe3fab1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:14.117350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780ffd-b5ea-461d-99c1-8e6d2fe3fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:14.117817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780ffd-b5ea-461d-99c1-8e6d2fe3fab1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:14.118664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90780ffd-b5ea-461d-99c1-8e6d2fe3fab1 None admin] RBAC: Adding query filter params (interface=public, service_id=94fbfa37f2bd4a89a76889cc91edb9c1, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:14.119122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90780ffd-b5ea-461d-99c1-8e6d2fe3fab1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:14.120713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90780ffd-b5ea-461d-99c1-8e6d2fe3fab1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:14.127644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 480/943] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:45:13 2022] GET /identity/v3/endpoints?service_id=94fbfa37f2bd4a89a76889cc91edb9c1&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:14.134727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:14.248969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456efe-3e56-4147-b90b-660d59723e6f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:14.283695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34375942-99e5-4848-a249-8d6bb2350597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:14.284686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34375942-99e5-4848-a249-8d6bb2350597 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:14.285138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34375942-99e5-4848-a249-8d6bb2350597 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:14.285685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34375942-99e5-4848-a249-8d6bb2350597 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:14.286456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34375942-99e5-4848-a249-8d6bb2350597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:14.288025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34375942-99e5-4848-a249-8d6bb2350597 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:14.294647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 464/944] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:45:14 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:45:14.754890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd558c-502f-4c8c-ba00-2c4d6d4b0ec3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:14.755404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd558c-502f-4c8c-ba00-2c4d6d4b0ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:14.755404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd558c-502f-4c8c-ba00-2c4d6d4b0ec3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:14.757027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 481/945] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:14.762875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c1d4-0141-4347-bdb4-1d60ee4771b9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:14.763289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c1d4-0141-4347-bdb4-1d60ee4771b9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:14.763429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c1d4-0141-4347-bdb4-1d60ee4771b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:14.781740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f639c1d4-0141-4347-bdb4-1d60ee4771b9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:14.782916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f639c1d4-0141-4347-bdb4-1d60ee4771b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:14.836332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:14.855444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 465/946] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:15.120620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd558c-502f-4c8c-ba00-2c4d6d4b0ec3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:15.226995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd558c-502f-4c8c-ba00-2c4d6d4b0ec3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:15.263995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:15.264682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:15.264905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:15.265175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:15.265875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:15.267456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:15.270220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING keystone.server.flask.common [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 14:45:15.290973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.sql.core [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 14:45:15.290973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 14:45:15.290973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [parameters: {'id': 'ab432918cadb4c0bbcad4ce8e7998cb8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 14:45:15.290973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81444) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c67eca1b-f5f6-4be1-8ade-e84e091da9ec None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ab432918cadb4c0bbcad4ce8e7998cb8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ab432918cadb4c0bbcad4ce8e7998cb8. Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application conn.query(q) Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 14:45:15.298676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result.read() Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 14:45:15.299617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application next(self.gen) Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application yield resource Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application next(self.gen) Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise self.value Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application session.commit() Sep 09 14:45:15.300726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.session.flush() Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application compat.raise_( Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 14:45:15.301699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 14:45:15.302617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application conn.query(q) Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result.read() Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 14:45:15.303462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [parameters: {'id': 'ab432918cadb4c0bbcad4ce8e7998cb8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:15.304353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:15.306484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ab432918cadb4c0bbcad4ce8e7998cb8. Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.309022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 482/947] 10.222.0.15 () {60 vars in 1140 bytes} [Fri Sep 9 14:45:15 2022] POST /identity/v3/projects => generated 262 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:15.310521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c1d4-0141-4347-bdb4-1d60ee4771b9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:15.373515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245572c-375a-4d0b-98e0-db4fd2442086 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:15.373783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245572c-375a-4d0b-98e0-db4fd2442086 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:15.374056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245572c-375a-4d0b-98e0-db4fd2442086 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:15.378426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 483/948] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:15.384152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1e2d1-8b81-4399-a74b-ed9da0034685 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:15.384435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1e2d1-8b81-4399-a74b-ed9da0034685 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:15.384707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1e2d1-8b81-4399-a74b-ed9da0034685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:15.408911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7da1e2d1-8b81-4399-a74b-ed9da0034685 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:15.411091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7da1e2d1-8b81-4399-a74b-ed9da0034685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:15.422676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c1d4-0141-4347-bdb4-1d60ee4771b9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:15.446356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:15.464001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:15.465061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:15.465562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:15.466198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:15.473999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:15.475762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:15.487985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:15.488861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:15.490121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:15.492306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 466/949] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:15.501244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:15.501398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 484/950] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:15.547566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:15.603138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffdf38-0f5e-4b27-bbec-cdba1df90933 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:15.638532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:15.639130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:15.639404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:15.639669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:15.640322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:15.640671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:15.642151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:15.665363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 467/951] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:45:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:15.779278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1e2d1-8b81-4399-a74b-ed9da0034685 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:15.890614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1e2d1-8b81-4399-a74b-ed9da0034685 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:15.926227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:15.926860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:15.927140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:15.927413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:15.945898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:15.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:15.951625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:15.952438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 485/952] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 14:45:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:15.959866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.073893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dedb6b-b352-4ebe-b1be-eb8cbcb3e998 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:16.076254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:16.108128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:16.108732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.109009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.109299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.109961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:16.110255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:16.111605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:16.120916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 468/953] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 14:45:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:16.130378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.157668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:16.245129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b29555-e743-4a6a-aad7-c969ea6027ec None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:16.279933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f740de-4959-4e5b-8b65-2227d5fc9494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:16.280562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f740de-4959-4e5b-8b65-2227d5fc9494 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.280841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f740de-4959-4e5b-8b65-2227d5fc9494 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.281113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f740de-4959-4e5b-8b65-2227d5fc9494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.292296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f740de-4959-4e5b-8b65-2227d5fc9494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:16.293936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f740de-4959-4e5b-8b65-2227d5fc9494 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b1f740de-4959-4e5b-8b65-2227d5fc9494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:16.309754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:16.310765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:16.311795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:16.311795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:16.311795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:16.311795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 486/954] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:16.316978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.322873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c956e57-e2df-4d8a-ae84-672b5dbd09fa None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.323157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c956e57-e2df-4d8a-ae84-672b5dbd09fa None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.323430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c956e57-e2df-4d8a-ae84-672b5dbd09fa None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.324894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 487/955] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:16.329699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e133e-20f3-4208-99ea-fa518b355f88 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.329989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e133e-20f3-4208-99ea-fa518b355f88 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.330265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e133e-20f3-4208-99ea-fa518b355f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.354755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f11e133e-20f3-4208-99ea-fa518b355f88 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:16.356484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f11e133e-20f3-4208-99ea-fa518b355f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:16.439974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7706dc5-3c9f-421b-af6d-aeca1ec24d3a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:16.442918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 488/956] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:16.487258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773d8ba-c40a-4162-884c-3ba4e69aa83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:16.487875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773d8ba-c40a-4162-884c-3ba4e69aa83e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.488157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773d8ba-c40a-4162-884c-3ba4e69aa83e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.488431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773d8ba-c40a-4162-884c-3ba4e69aa83e None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.489080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773d8ba-c40a-4162-884c-3ba4e69aa83e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:16.490203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773d8ba-c40a-4162-884c-3ba4e69aa83e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:16.492146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773d8ba-c40a-4162-884c-3ba4e69aa83e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:16.505060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 469/957] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:16.510482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e133e-20f3-4208-99ea-fa518b355f88 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.519879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:16.576311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c6503-cb10-4171-9d15-091a05b5b1b0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.576694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c6503-cb10-4171-9d15-091a05b5b1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.576966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c6503-cb10-4171-9d15-091a05b5b1b0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.579590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 470/958] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:16.583488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fda98c-39c6-4886-9fd2-63cf0568c801 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.583763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fda98c-39c6-4886-9fd2-63cf0568c801 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.584035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fda98c-39c6-4886-9fd2-63cf0568c801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.607090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-40fda98c-39c6-4886-9fd2-63cf0568c801 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:16.608287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-40fda98c-39c6-4886-9fd2-63cf0568c801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:16.621486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e133e-20f3-4208-99ea-fa518b355f88 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:16.633066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:16.658868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:16.659816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.660240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.660656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.669728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:16.671563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:16.684963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:16.686180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 14:45:16.687284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:16.689151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 489/959] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:45:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:16.693852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 471/960] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:16.695396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.748436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fda98c-39c6-4886-9fd2-63cf0568c801 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.767359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:16.805021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8963c9-a85d-4025-ade2-71d87a6a1084 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:16.841040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:16.842044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.842510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.843019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.843912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:16.844404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:16.846122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:16.864389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fda98c-39c6-4886-9fd2-63cf0568c801 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:16.869356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 490/961] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 14:45:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:16.870433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.908380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:16.909017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:16.909334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:16.909610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:16.910194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:16.911559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:16.923772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:16.924810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:16.925690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:16.930265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 472/962] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:16.931885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:16.957499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:16.988660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f0a10-e5d2-49d7-9233-81281490f638 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.019464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:17.028318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.028929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.029233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.029508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.047080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f369a-29e1-47a0-93a1-1baa2fa477c3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.090582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c342d26-519f-4809-a672-420537075c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.091520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c342d26-519f-4809-a672-420537075c6a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.091802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c342d26-519f-4809-a672-420537075c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.092075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c342d26-519f-4809-a672-420537075c6a None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.092700 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c342d26-519f-4809-a672-420537075c6a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:17.093001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c342d26-519f-4809-a672-420537075c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.095100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c342d26-519f-4809-a672-420537075c6a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.096637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.098118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.104954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 473/963] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:45:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:17.104954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c342d26-519f-4809-a672-420537075c6a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.195193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:17.201807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 491/964] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:16 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:45:17.202896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.218728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c342d26-519f-4809-a672-420537075c6a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.229545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:17.255444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.256884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.257168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.257463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.258042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.259391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:17.268402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:17.269427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:17.270258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 474/965] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:17.271073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.319363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d7a42-0001-4638-baab-00cf4f19150e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.354404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.355010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.355285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.355556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.356179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:17.356478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.357875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.373233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 492/966] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:17.373233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.388035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc40d6-cabe-44b5-8495-cc7d84b1da3a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.433947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.435440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.435868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.436216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.452647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.454827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.468381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:17.478641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:17.479633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:17.480621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:17.480621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:17.480621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:17.485317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 475/967] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:17.485744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:17.486625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.501903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc6295-fb70-4d47-82e4-8fe6855f4bc9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.543474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.544178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.544460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.544731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.545844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:17.546151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.547859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.555323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 493/968] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:17.561921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.624570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b1eaf-46cc-4d6e-b48d-2b06c09c63d9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.660608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.661366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.661716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.662057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.663294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:17.663662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.665156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.673524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2c2cf-8dff-45d7-9eee-bfbc74bc62ef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.677804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 476/969] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:17.685703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.713627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.714604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.715089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.715580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.716423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:17.716977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.718575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.724708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 494/970] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:17.733332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.748877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:17.753779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:17.808281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd57927-a9aa-4594-9cfb-ba6a7d923080 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.842285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325c44a-daac-4f8d-aff4-b5c44c3deeb9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:17.862742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.864131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.864597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.865014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.876256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.878945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.883109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4fbb0-d353-4bbd-b092-96a848a24d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:17.883835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4fbb0-d353-4bbd-b092-96a848a24d73 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.884198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4fbb0-d353-4bbd-b092-96a848a24d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.884571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4fbb0-d353-4bbd-b092-96a848a24d73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.885384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4fbb0-d353-4bbd-b092-96a848a24d73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:17.887096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4fbb0-d353-4bbd-b092-96a848a24d73 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:17.892162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:17.893042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:17.893972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:17.895449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 477/971] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:17.899644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:17.925228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 495/972] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:45:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:17.961470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5525b-3127-4f92-9ad6-d09f02c68e26 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.961895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5525b-3127-4f92-9ad6-d09f02c68e26 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.962266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5525b-3127-4f92-9ad6-d09f02c68e26 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.963803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 496/973] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:17.969501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c0154-34e8-41c9-a2a6-860033c6fd11 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:17.969939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c0154-34e8-41c9-a2a6-860033c6fd11 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:17.970409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c0154-34e8-41c9-a2a6-860033c6fd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:17.990424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2b2c0154-34e8-41c9-a2a6-860033c6fd11 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:17.992398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2b2c0154-34e8-41c9-a2a6-860033c6fd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:18.031165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea78340-723b-4385-9095-a9a677cc9b25 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:18.055741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:18.084563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:18.087203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:18.088188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:18.088682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:18.089373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:18.090333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:18.090990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:18.092672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 497/974] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:18.093938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:18.110124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 478/975] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:18.117260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c0154-34e8-41c9-a2a6-860033c6fd11 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:18.227949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c0154-34e8-41c9-a2a6-860033c6fd11 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:18.264695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:18.265708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:18.266212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:18.266686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:18.267641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] RBAC: Adding query filter params (role.id=e5c78c8957b941eaba90da16737d4525, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=b3e63489796544afaa44deab302681b9) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:18.268190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:18.269878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:18.276394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 498/976] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:18 2022] GET /identity/v3/role_assignments?user.id=b3e63489796544afaa44deab302681b9&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=e5c78c8957b941eaba90da16737d4525 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:18.362949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:18.472228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd5e46-5047-4f7d-85e0-e913446152f4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:18.514117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:18.515110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:18.515533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:18.515937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:18.516747 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:18.518571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:18.520824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING keystone.server.flask.common [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 14:45:18.535542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.sql.core [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 14:45:18.535542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 14:45:18.535542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [parameters: {'user_id': '4c69ba430c2c4b9e8851af5a6eecac21', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 14:45:18.535542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81443) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application conn.query(q) Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result.read() Sep 09 14:45:18.552914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 14:45:18.554475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application next(self.gen) Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application yield resource Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application next(self.gen) Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise self.value Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application session.commit() Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 14:45:18.555434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.session.flush() Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application compat.raise_( Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 14:45:18.556343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 14:45:18.557312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application conn.query(q) Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result.read() Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 14:45:18.558168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [parameters: {'user_id': '4c69ba430c2c4b9e8851af5a6eecac21', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:18.559061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 14:45:18.559845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 14:45:18.560802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 14:45:18.560802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 14:45:18.560802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 14:45:18.560802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 14:45:18.560802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 14:45:18.560802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:18.560802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 479/977] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 14:45:18 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:18.563730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:18.630008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:18.673596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f3de3-efa4-4840-9028-70b85fd80709 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:18.713379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c66fa-183d-436e-9f09-78a4af8b9846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:18.714016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c66fa-183d-436e-9f09-78a4af8b9846 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:18.714295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c66fa-183d-436e-9f09-78a4af8b9846 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:18.714567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c66fa-183d-436e-9f09-78a4af8b9846 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:18.722818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c66fa-183d-436e-9f09-78a4af8b9846 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:18.724292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c66fa-183d-436e-9f09-78a4af8b9846 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-839c66fa-183d-436e-9f09-78a4af8b9846 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:18.737658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:18.738918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:18.740202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 499/978] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:45:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:18.745780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:18.790358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:18.865250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741131f-6827-4b0a-a165-d48fb017da64 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:18.903947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070533ec-86ee-4d68-9559-e109e30e21a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:18.904560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070533ec-86ee-4d68-9559-e109e30e21a1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:18.904835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070533ec-86ee-4d68-9559-e109e30e21a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:18.905110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070533ec-86ee-4d68-9559-e109e30e21a1 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:18.905841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070533ec-86ee-4d68-9559-e109e30e21a1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:18.906140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070533ec-86ee-4d68-9559-e109e30e21a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:18.907760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070533ec-86ee-4d68-9559-e109e30e21a1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:18.949034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 480/979] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 14:45:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:19.644070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695058-a4a2-4e74-b336-c21c96cc0d50 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:19.644613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695058-a4a2-4e74-b336-c21c96cc0d50 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:19.644613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695058-a4a2-4e74-b336-c21c96cc0d50 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:19.646122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 500/980] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:19.651776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8978e2f-ee7a-47c2-aaf2-c54ca1bff918 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:19.652176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8978e2f-ee7a-47c2-aaf2-c54ca1bff918 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:19.652353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8978e2f-ee7a-47c2-aaf2-c54ca1bff918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:19.671323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c8978e2f-ee7a-47c2-aaf2-c54ca1bff918 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:19.672855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c8978e2f-ee7a-47c2-aaf2-c54ca1bff918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:19.741053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 481/981] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:20.012432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695058-a4a2-4e74-b336-c21c96cc0d50 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:20.121699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695058-a4a2-4e74-b336-c21c96cc0d50 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:20.145086 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:20.158139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f4901-935c-464c-84bf-e3b8333e2769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:20.158744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f4901-935c-464c-84bf-e3b8333e2769 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:20.159022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f4901-935c-464c-84bf-e3b8333e2769 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:20.159290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f4901-935c-464c-84bf-e3b8333e2769 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:20.159933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f4901-935c-464c-84bf-e3b8333e2769 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:20.161324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f4901-935c-464c-84bf-e3b8333e2769 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:20.179051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 501/982] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:45:20 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:20.203517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee32a3c-51c1-4f9e-bbdc-132aaaccc57a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:20.203836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee32a3c-51c1-4f9e-bbdc-132aaaccc57a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:20.204061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee32a3c-51c1-4f9e-bbdc-132aaaccc57a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:20.205704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 482/983] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:20.212291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29b244-4dea-43e1-89e0-84df879dce40 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:20.212577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29b244-4dea-43e1-89e0-84df879dce40 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:20.212874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29b244-4dea-43e1-89e0-84df879dce40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:20.232124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7d29b244-4dea-43e1-89e0-84df879dce40 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:20.233398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7d29b244-4dea-43e1-89e0-84df879dce40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:20.316556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 502/984] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:20.572603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373226b-7a60-47dd-81ba-40c9a7b1eb0d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:20.572785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373226b-7a60-47dd-81ba-40c9a7b1eb0d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:20.573056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373226b-7a60-47dd-81ba-40c9a7b1eb0d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:20.574437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 483/985] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:20.580054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12458c13-79b2-4cef-b752-23052f6824cc None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:20.580340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12458c13-79b2-4cef-b752-23052f6824cc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:20.580608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12458c13-79b2-4cef-b752-23052f6824cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:20.586093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373226b-7a60-47dd-81ba-40c9a7b1eb0d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:20.599122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-12458c13-79b2-4cef-b752-23052f6824cc None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:20.600333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-12458c13-79b2-4cef-b752-23052f6824cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:20.615896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:20.677559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 503/986] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:20.696048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373226b-7a60-47dd-81ba-40c9a7b1eb0d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:20.729887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:20.730515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:20.730791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:20.731060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:20.731692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:20.733045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:20.742438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:20.743506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:20.744348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 484/987] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:20.749554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12458c13-79b2-4cef-b752-23052f6824cc None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:20.789728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:20.850981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12458c13-79b2-4cef-b752-23052f6824cc None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:20.884977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:20.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:20.885905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:20.886173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:20.887105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:20.887400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:20.888767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:20.898625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 504/988] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:20.899892 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:20.938270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:20.999186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:21.012484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d3ec0-fdb7-4600-8ffb-07dd6000f1a7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:21.046291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb750da9-3cc5-4328-b57b-796a1041feb6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:21.049256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:21.052542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:21.053172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:21.053479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:21.053749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:21.054377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:21.054672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:21.056062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:21.060900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 485/989] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:21.066846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:21.088498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:21.089442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:21.089717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:21.089983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:21.090550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:21.091917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:21.101498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:21.102562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:21.103427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 505/990] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 9 14:45:20 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:21.108904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:21.174318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f8dd2-47b8-4c8e-8048-7e226b081e88 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:21.208013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:21.208616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:21.208893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:21.209698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:21.210425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] RBAC: Adding query filter params (interface=admin, service_id=94fbfa37f2bd4a89a76889cc91edb9c1, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:21.210702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:21.212093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:21.212963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae95887-8acd-48da-aaf6-f7f12b3c57a1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:21.217105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 486/991] 10.222.0.15 () {56 vars in 1249 bytes} [Fri Sep 9 14:45:21 2022] GET /identity/v3/endpoints?service_id=94fbfa37f2bd4a89a76889cc91edb9c1&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:21.222845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:21.241778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:21.255498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:21.256124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:21.256552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:21.256828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:21.258199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] RBAC: Adding query filter params (name=image) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:21.258500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:21.259857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:21.265016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 506/992] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:21.270522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:21.301060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:21.326372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6399a72-772c-4124-91f9-8a3ab0d913c0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:21.362487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9192d2-a781-4aac-85ef-064da41e0f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:21.363386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9192d2-a781-4aac-85ef-064da41e0f0d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:21.363822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9192d2-a781-4aac-85ef-064da41e0f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:21.364096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9192d2-a781-4aac-85ef-064da41e0f0d None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:21.365011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9192d2-a781-4aac-85ef-064da41e0f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:21.366567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9192d2-a781-4aac-85ef-064da41e0f0d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:21.373003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 487/993] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:45:21 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:45:21.381874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cee41c-503b-425b-8953-9763ab2d48de None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:21.421081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f4fed-139e-4d0c-a7dd-8657852a1051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:21.421737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4fed-139e-4d0c-a7dd-8657852a1051 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:21.422018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4fed-139e-4d0c-a7dd-8657852a1051 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:21.422289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4fed-139e-4d0c-a7dd-8657852a1051 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:21.422932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f4fed-139e-4d0c-a7dd-8657852a1051 None admin] RBAC: Adding query filter params (type=image) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:21.423233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f4fed-139e-4d0c-a7dd-8657852a1051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:21.424774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f4fed-139e-4d0c-a7dd-8657852a1051 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:21.433609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 507/994] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:22.414422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3193f-2122-43b9-a90c-1a4282911cdd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:22.414957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3193f-2122-43b9-a90c-1a4282911cdd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:22.414957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3193f-2122-43b9-a90c-1a4282911cdd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:22.416322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 488/995] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:22.421560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e39af4-2829-4962-8e86-40c497fe4155 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:22.421971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e39af4-2829-4962-8e86-40c497fe4155 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:22.422147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e39af4-2829-4962-8e86-40c497fe4155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:22.440500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a9e39af4-2829-4962-8e86-40c497fe4155 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:22.441730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a9e39af4-2829-4962-8e86-40c497fe4155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:22.510056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 508/996] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:22.783380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3193f-2122-43b9-a90c-1a4282911cdd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:22.865143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:22.885283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3193f-2122-43b9-a90c-1a4282911cdd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:22.918269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:22.918889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:22.919164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:22.919430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:22.919998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:22.921421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:22.930076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:22.931052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:22.931890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:22.936212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 489/997] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:22.937035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e39af4-2829-4962-8e86-40c497fe4155 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:22.962735 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:23.042688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e39af4-2829-4962-8e86-40c497fe4155 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:23.076279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:23.076880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.077228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.077514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.078134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:23.078428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:23.079787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:23.085233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 509/998] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:45:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:23.090639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:23.188989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf1e71-5da7-4513-998d-e829aab80dc3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:23.223093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:23.223699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.223973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.224297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.224983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:23.225357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:23.226708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:23.232221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 490/999] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:45:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:23.240792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:23.328674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:23.342601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11eafc-348d-45ee-9a05-247544395d62 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:23.375559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421a65f-0de3-4dae-b001-5c59ac55c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:23.376205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421a65f-0de3-4dae-b001-5c59ac55c204 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.376483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421a65f-0de3-4dae-b001-5c59ac55c204 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.376750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421a65f-0de3-4dae-b001-5c59ac55c204 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.377461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421a65f-0de3-4dae-b001-5c59ac55c204 None admin] RBAC: Adding query filter params (interface=public, service_id=7a54efba5bff43418c6379a76a398504, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:23.377764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421a65f-0de3-4dae-b001-5c59ac55c204 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:23.379122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421a65f-0de3-4dae-b001-5c59ac55c204 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:23.389018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 510/1000] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:45:23 2022] GET /identity/v3/endpoints?service_id=7a54efba5bff43418c6379a76a398504&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:23.390523 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:23.403307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:23.500083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2076a-d9c4-4187-af8c-ac1fc27268d5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:23.535904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2cf78-54eb-4eb7-a82e-e92f556868ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:23.536511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2cf78-54eb-4eb7-a82e-e92f556868ea None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.536786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2cf78-54eb-4eb7-a82e-e92f556868ea None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.537053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2cf78-54eb-4eb7-a82e-e92f556868ea None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.538289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2cf78-54eb-4eb7-a82e-e92f556868ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:23.539649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2cf78-54eb-4eb7-a82e-e92f556868ea None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:23.553239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 491/1001] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:45:23 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:45:23.579766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8ac55-cbd4-424c-9d7f-32c4b2b164f2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.580266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8ac55-cbd4-424c-9d7f-32c4b2b164f2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.580266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8ac55-cbd4-424c-9d7f-32c4b2b164f2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.581804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 511/1002] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:23.587961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b99b2-cdcf-4382-bb73-6028722dee37 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.588282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b99b2-cdcf-4382-bb73-6028722dee37 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.588508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b99b2-cdcf-4382-bb73-6028722dee37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.597253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7b54-4a81-4314-87fd-0aea9d5a3187 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.597574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7b54-4a81-4314-87fd-0aea9d5a3187 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.597801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7b54-4a81-4314-87fd-0aea9d5a3187 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.599381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 512/1003] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:23.605615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e13cf-6eaa-44d9-9b6e-4459c35d2be0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:23.605939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e13cf-6eaa-44d9-9b6e-4459c35d2be0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:23.606168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e13cf-6eaa-44d9-9b6e-4459c35d2be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:23.608928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a55b99b2-cdcf-4382-bb73-6028722dee37 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:23.610214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a55b99b2-cdcf-4382-bb73-6028722dee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:23.624986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-794e13cf-6eaa-44d9-9b6e-4459c35d2be0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:23.626354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-794e13cf-6eaa-44d9-9b6e-4459c35d2be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:23.686467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 492/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:23.701239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 513/1005] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:23.951312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b99b2-cdcf-4382-bb73-6028722dee37 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:23.955480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:23.970176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e13cf-6eaa-44d9-9b6e-4459c35d2be0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:24.061916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:24.062119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b99b2-cdcf-4382-bb73-6028722dee37 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:24.082682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e13cf-6eaa-44d9-9b6e-4459c35d2be0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:24.099405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:24.100041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:24.100553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:24.100837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:24.101784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:24.103157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:24.114987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:24.116490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:24.117368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 493/1006] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 14:45:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:24.120815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:24.121458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:24.122304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:24.122890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:24.123161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:24.123801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:24.125175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:24.145240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 514/1007] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:45:23 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:24.200748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:24.239152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9aff8-2f2a-4776-bb34-573bda433c54 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:24.283818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:24.284451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:24.284726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:24.284993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:24.285662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:24.285965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:24.287368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:24.297748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 494/1008] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:24.298889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:24.401352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ce23d-fd2d-4333-8852-5c1fef9a9020 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:24.435127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e916f8-46a1-4912-a4cd-ca18f2e7293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:24.435757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e916f8-46a1-4912-a4cd-ca18f2e7293d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:24.436033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e916f8-46a1-4912-a4cd-ca18f2e7293d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:24.436303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e916f8-46a1-4912-a4cd-ca18f2e7293d None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:24.436954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e916f8-46a1-4912-a4cd-ca18f2e7293d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:24.437288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e916f8-46a1-4912-a4cd-ca18f2e7293d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:24.438720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e916f8-46a1-4912-a4cd-ca18f2e7293d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:24.448933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 515/1009] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 14:45:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:24.450212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:24.554522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2ea71-710e-4dad-ae82-82c9e0e47952 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:24.579883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:24.588932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9840602-d30f-4903-8ae7-b0976a7ab469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:24.589571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840602-d30f-4903-8ae7-b0976a7ab469 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:24.589852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840602-d30f-4903-8ae7-b0976a7ab469 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:24.590122 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840602-d30f-4903-8ae7-b0976a7ab469 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:24.590791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9840602-d30f-4903-8ae7-b0976a7ab469 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:24.592140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9840602-d30f-4903-8ae7-b0976a7ab469 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:24.617228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 495/1010] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:45:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:25.715031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3eb26-83f3-41c1-989b-fa8e0623e8c6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:25.715590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3eb26-83f3-41c1-989b-fa8e0623e8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:25.715590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3eb26-83f3-41c1-989b-fa8e0623e8c6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:25.716925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 516/1011] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:25.722132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632450b-3240-48f8-b344-4c015ac22665 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:25.722532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632450b-3240-48f8-b344-4c015ac22665 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:25.722694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632450b-3240-48f8-b344-4c015ac22665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:25.741167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c632450b-3240-48f8-b344-4c015ac22665 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:25.742354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c632450b-3240-48f8-b344-4c015ac22665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:25.813807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 496/1012] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:25 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:26.080709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3eb26-83f3-41c1-989b-fa8e0623e8c6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:26.090248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:26.184118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3eb26-83f3-41c1-989b-fa8e0623e8c6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:26.221139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f490f-9908-449a-91a7-6ede0d4575f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:26.221857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f490f-9908-449a-91a7-6ede0d4575f4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.222132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f490f-9908-449a-91a7-6ede0d4575f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.222396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f490f-9908-449a-91a7-6ede0d4575f4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.222975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f490f-9908-449a-91a7-6ede0d4575f4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:26.224363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f490f-9908-449a-91a7-6ede0d4575f4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d46f490f-9908-449a-91a7-6ede0d4575f4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:26.234209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:26.235232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 14:45:26.236069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:26.240989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 517/1013] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:26.242035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632450b-3240-48f8-b344-4c015ac22665 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:26.315223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9adff-9e42-4984-8cf6-735f2f80cc21 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.315413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9adff-9e42-4984-8cf6-735f2f80cc21 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.315674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9adff-9e42-4984-8cf6-735f2f80cc21 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.317028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 518/1014] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:26.322457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe595f-e05d-4cd3-a1e5-8d4d7e313a24 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.322764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe595f-e05d-4cd3-a1e5-8d4d7e313a24 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.323042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe595f-e05d-4cd3-a1e5-8d4d7e313a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.341514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f2fe595f-e05d-4cd3-a1e5-8d4d7e313a24 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:26.342728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f2fe595f-e05d-4cd3-a1e5-8d4d7e313a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:26.348284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632450b-3240-48f8-b344-4c015ac22665 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:26.363432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:26.386412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:26.387301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.387580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.387848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.388469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:26.388768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:26.390162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:26.395166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 497/1015] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:45:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:26.401582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:26.416384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:26.421020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 519/1016] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:26.503223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03e57-788c-4a86-9382-8b94a0926afc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:26.536443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:26.537039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.537976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.538248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.538858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:26.539151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:26.540488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:26.545288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 498/1017] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 14:45:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:26.550837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe595f-e05d-4cd3-a1e5-8d4d7e313a24 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:26.649053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe595f-e05d-4cd3-a1e5-8d4d7e313a24 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:26.685671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fcb73-3ca0-416f-91d6-e0709a5abf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:26.686274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fcb73-3ca0-416f-91d6-e0709a5abf99 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.686555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fcb73-3ca0-416f-91d6-e0709a5abf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.686819 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fcb73-3ca0-416f-91d6-e0709a5abf99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.687492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fcb73-3ca0-416f-91d6-e0709a5abf99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:26.688846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fcb73-3ca0-416f-91d6-e0709a5abf99 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:26.691695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:26.721228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 520/1018] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:45:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:26.752144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:26.770130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559087b-7699-403b-b1b6-306d734d6e07 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.770489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559087b-7699-403b-b1b6-306d734d6e07 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.770682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559087b-7699-403b-b1b6-306d734d6e07 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.772976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 521/1019] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:26.778844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbccaf-621e-419c-b663-417c54114846 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.779124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbccaf-621e-419c-b663-417c54114846 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.779400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbccaf-621e-419c-b663-417c54114846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.799770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d9cbccaf-621e-419c-b663-417c54114846 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:26.801025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d9cbccaf-621e-419c-b663-417c54114846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:26.820255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48eaa4-c676-43d2-ab29-9ce9306a0de7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:26.861636 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:26.867403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:26.868301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:26.868836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:26.869289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] PATH_INFO: `/v3/services/image` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:26.870024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:26.871600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:26.881589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:26.882761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:26.883772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:26.884512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 522/1020] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:45:26.890796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 499/1021] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 9 14:45:26 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:26.891846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbccaf-621e-419c-b663-417c54114846 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:26.998858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbccaf-621e-419c-b663-417c54114846 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:27.038862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:27.039468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:27.039744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:27.040010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:27.040635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] RBAC: Adding query filter params (name=image) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:27.040932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:27.042320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:27.047366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 523/1022] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:27.053858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:27.150099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d43fe1-f68a-4744-b755-9a26fc45c40d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:27.151634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:27.162497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:27.187859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:27.188467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:27.188740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:27.189008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:27.189656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] RBAC: Adding query filter params (type=image) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:27.189954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:27.191311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:27.196805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 500/1023] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:27.202875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:27.218147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:27.271628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017d74-b063-47dd-9d37-3b5dd8469d9a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:27.306608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:27.307235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:27.307509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:27.307778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:27.308862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:27.310331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:27.314882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf7fde-95ce-4080-8426-9d3508ff948f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:27.316750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 524/1024] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:27.317520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:27.350060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:27.350934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:27.351211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:27.351478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:27.352171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] RBAC: Adding query filter params (interface=public, service_id=a78268f8934d4ea3b7575f9f0abd22ae, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:27.352473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:27.353842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:27.358948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 501/1025] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:45:27 2022] GET /identity/v3/endpoints?service_id=a78268f8934d4ea3b7575f9f0abd22ae&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:27.364573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:27.398546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:27.421860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9415d-1148-435e-a998-87ae120a1ee6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:27.440309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:27.458722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536a7ec-c68f-4c2e-984b-ef8d41dd7999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:27.459366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536a7ec-c68f-4c2e-984b-ef8d41dd7999 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:27.459643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536a7ec-c68f-4c2e-984b-ef8d41dd7999 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:27.459911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536a7ec-c68f-4c2e-984b-ef8d41dd7999 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:27.460556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536a7ec-c68f-4c2e-984b-ef8d41dd7999 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:27.462013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536a7ec-c68f-4c2e-984b-ef8d41dd7999 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:27.476563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86291c4-6a34-471c-a7af-7c9ac3365c3f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:27.491842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 525/1026] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:45:27 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:27.515177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553517ae-7259-4c8d-931c-b03ab73cd4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:27.515804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553517ae-7259-4c8d-931c-b03ab73cd4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:27.516082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553517ae-7259-4c8d-931c-b03ab73cd4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:27.516352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553517ae-7259-4c8d-931c-b03ab73cd4d9 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:27.516967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553517ae-7259-4c8d-931c-b03ab73cd4d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:27.518435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553517ae-7259-4c8d-931c-b03ab73cd4d9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:27.533583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 502/1027] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:45:27 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:45:28.989870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de852b-8aa2-46ba-8244-965adc324aa1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:28.990967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de852b-8aa2-46ba-8244-965adc324aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:28.991458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de852b-8aa2-46ba-8244-965adc324aa1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:28.993274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 526/1028] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:28.998057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7400a-ffaf-4fd0-96cb-efa3172e8b7e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:28.998457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7400a-ffaf-4fd0-96cb-efa3172e8b7e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:28.998620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7400a-ffaf-4fd0-96cb-efa3172e8b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.018246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e8f7400a-ffaf-4fd0-96cb-efa3172e8b7e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:29.018974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e8f7400a-ffaf-4fd0-96cb-efa3172e8b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:29.087222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 503/1029] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:29.357937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de852b-8aa2-46ba-8244-965adc324aa1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:29.463354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de852b-8aa2-46ba-8244-965adc324aa1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:29.498000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea452f7-79f2-44f7-a4b4-9786fd879363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:29.498983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea452f7-79f2-44f7-a4b4-9786fd879363 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:29.499490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea452f7-79f2-44f7-a4b4-9786fd879363 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:29.499960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea452f7-79f2-44f7-a4b4-9786fd879363 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.511001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea452f7-79f2-44f7-a4b4-9786fd879363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8ea452f7-79f2-44f7-a4b4-9786fd879363 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:29.516426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:29.517391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:29.518273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:29.521359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 527/1030] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 14:45:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:29.524751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7400a-ffaf-4fd0-96cb-efa3172e8b7e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:29.550810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:29.638759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7400a-ffaf-4fd0-96cb-efa3172e8b7e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:29.672837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:29.673476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:29.673764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:29.674036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.674694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:29.674995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:29.676352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:29.685245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 504/1031] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 14:45:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:29.689839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f952bce-4d2b-4500-b432-5aec90c28078 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:29.690355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f952bce-4d2b-4500-b432-5aec90c28078 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:29.690631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:29.691058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f952bce-4d2b-4500-b432-5aec90c28078 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.692679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 528/1032] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:29.694118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e26b2-b0c6-4936-b0a8-48121e7ab820 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:29.695273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e26b2-b0c6-4936-b0a8-48121e7ab820 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:29.695562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e26b2-b0c6-4936-b0a8-48121e7ab820 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.696748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 529/1033] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:29.697792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3303fae-fe4e-43e6-be62-0c9301c1e608 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:29.698068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3303fae-fe4e-43e6-be62-0c9301c1e608 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:29.698336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3303fae-fe4e-43e6-be62-0c9301c1e608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.716859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b3303fae-fe4e-43e6-be62-0c9301c1e608 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:29.718068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b3303fae-fe4e-43e6-be62-0c9301c1e608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:29.741759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:29.774834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:29.786678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 530/1034] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:29.788546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d140f13-321d-47ec-9f29-ce5eeef7b763 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:29.788825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d140f13-321d-47ec-9f29-ce5eeef7b763 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:29.789093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d140f13-321d-47ec-9f29-ce5eeef7b763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.796112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3bf7-0c47-4f01-b09f-0d33505415d8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:29.806858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8d140f13-321d-47ec-9f29-ce5eeef7b763 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:29.808227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8d140f13-321d-47ec-9f29-ce5eeef7b763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:29.830067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:29.830687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:29.830962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:29.831230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:29.845832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:29.847482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:29.865726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:29.866695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:29.867581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:29.867581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:29.867581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:29.867581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 505/1035] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:29.872526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:29.885516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 531/1036] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:29.974087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d920ff-5b92-4b28-b6a9-aa0a07ffe794 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.006956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.007606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.007829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.008097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.008730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:30.015348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.015348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.024218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.034541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 506/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:30.036290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d140f13-321d-47ec-9f29-ce5eeef7b763 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.061566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.131415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.142847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d140f13-321d-47ec-9f29-ce5eeef7b763 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.167279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59ae1a-9b81-4e2e-88ac-4ecfdce2e313 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.178126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.178786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.179083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.179355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.186211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.187659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:30.198959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:30.199867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:30.200814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 532/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:30.201454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.201454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.201611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.201681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.201756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.205453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.206884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.213451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 507/1039] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:30.213451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.274736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.303888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b881311-1d6a-4946-9b33-3918d6715ce1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.311945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0ba4c-bef4-43bc-b435-1a123bf444f7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.336586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.337327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.337608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.337872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] PATH_INFO: `/v3/services/image` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.338439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.339801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.346195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.346794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.347071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.347341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.348055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:30.348358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:30.348470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:30.350176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:30.351189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 533/1040] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:30.354112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.354262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.366367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.369526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 508/1041] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:30.370224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.450103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e1823-5381-47e2-a252-31643fbdb172 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.467419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b93c1-544c-4f6f-9fdf-7a3b4c3a0631 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.483111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.483707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.483982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.484256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.484894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.486363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.490084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.501611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.502210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.502486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.502754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.503370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] RBAC: Adding query filter params (name=image) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:30.503667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.505005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.508410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 534/1042] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:45:30 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:30.509230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.510646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 509/1043] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:30.523654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.590567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.629925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014e54-cb8a-4752-940a-322898d34014 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.640147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553d8bf-28da-4178-b648-80bba957fe11 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.671234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.672286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.672784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.673260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.679828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.680447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.680729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.681003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.682262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] RBAC: Adding query filter params (type=image) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:30.682262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.683761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.690033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 510/1044] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:30.696719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.751219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.752676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.759858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.809161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870389f-7d51-4156-82db-b4a1e1d03f49 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:30.827369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:30.846746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:30.847770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:30.848070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:30.848344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:30.849379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:30.851389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:30.860507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 535/1045] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:45:30.866562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:30.877778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 511/1046] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:45:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:45:30.983624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b597e9-e948-4368-958a-dab52d23023e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:31.035532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:31.036181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:31.036456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:31.037004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:31.047477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:31.048943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:31.066508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:31.067482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:31.068934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:31.068934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:31.068934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:31.068934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 536/1047] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:31.073901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:31.179310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792710b-242e-4748-b2e9-916ff0cdffcc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:31.188033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:31.214110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:31.214714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:31.214990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:31.215255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:31.215878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:31.216176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:31.217547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:31.229825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 512/1048] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:31.235203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:31.254887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:31.336238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea38b3d-8001-40f1-a793-9b8f9d0a12f7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:31.370099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:31.370720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:31.370996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:31.371266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:31.377818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:31.379250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:31.390588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:31.391457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:31.392350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:31.396537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 537/1049] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:31.397515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:31.498001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7abc2e-b4e9-4f90-a27f-9800ec96556b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:31.530971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32eb01-478e-48a9-95c8-a076c70a7762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:31.531564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32eb01-478e-48a9-95c8-a076c70a7762 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:31.531837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32eb01-478e-48a9-95c8-a076c70a7762 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:31.532103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32eb01-478e-48a9-95c8-a076c70a7762 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:31.532778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32eb01-478e-48a9-95c8-a076c70a7762 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:31.533075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32eb01-478e-48a9-95c8-a076c70a7762 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:31.534521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32eb01-478e-48a9-95c8-a076c70a7762 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:31.546237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 513/1050] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:31.551979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:31.629123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:31.653313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77eb1e9-7534-42c4-b391-08848ed5a7a0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:31.688155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5956918-c8e6-405a-841c-2fefbfbbbc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:31.688988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5956918-c8e6-405a-841c-2fefbfbbbc03 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:31.689444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5956918-c8e6-405a-841c-2fefbfbbbc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:31.689865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5956918-c8e6-405a-841c-2fefbfbbbc03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:31.690726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5956918-c8e6-405a-841c-2fefbfbbbc03 None admin] RBAC: Adding query filter params (role.id=e5c78c8957b941eaba90da16737d4525, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=7b6809bf91d244e4bed5b8a256a3f95a) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:31.691204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5956918-c8e6-405a-841c-2fefbfbbbc03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:31.692776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5956918-c8e6-405a-841c-2fefbfbbbc03 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:31.698694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 538/1051] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:31 2022] GET /identity/v3/role_assignments?user.id=7b6809bf91d244e4bed5b8a256a3f95a&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=e5c78c8957b941eaba90da16737d4525 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:32.736565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd085e-26f4-4680-aba9-9fc65a14a0b7 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:32.737115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd085e-26f4-4680-aba9-9fc65a14a0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:32.737115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd085e-26f4-4680-aba9-9fc65a14a0b7 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:32.738512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 514/1052] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:32.743657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998642c-bced-4b64-8cb7-f1d874bdda07 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:32.744030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998642c-bced-4b64-8cb7-f1d874bdda07 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:32.744208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998642c-bced-4b64-8cb7-f1d874bdda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:32.763778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a998642c-bced-4b64-8cb7-f1d874bdda07 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:32.764981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a998642c-bced-4b64-8cb7-f1d874bdda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:32.833160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 539/1053] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:33.082177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee7ca-a4a1-4814-89e6-788616ab8b21 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.082506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee7ca-a4a1-4814-89e6-788616ab8b21 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.082733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee7ca-a4a1-4814-89e6-788616ab8b21 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.084147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 515/1054] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:33.089353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ab057-c6bf-427e-81f5-7a5f380b25cd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.089641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ab057-c6bf-427e-81f5-7a5f380b25cd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.089913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ab057-c6bf-427e-81f5-7a5f380b25cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.107242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee7ca-a4a1-4814-89e6-788616ab8b21 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:33.108001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f50ab057-c6bf-427e-81f5-7a5f380b25cd None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:33.109225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f50ab057-c6bf-427e-81f5-7a5f380b25cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:33.138279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:33.183972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 540/1055] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:33.216602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee7ca-a4a1-4814-89e6-788616ab8b21 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:33.250228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:33.250845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.251120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.251385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.262733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:33.267432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:33.268367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:33.270211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 516/1056] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:33.274516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ab057-c6bf-427e-81f5-7a5f380b25cd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:33.277089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:33.375404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ab057-c6bf-427e-81f5-7a5f380b25cd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:33.409987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:33.410587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.410864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.411132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.411783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:33.412117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:33.413505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:33.421720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 541/1057] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:33.427401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:33.476321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:33.522792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:33.537986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc039c-21cc-4b52-b1b4-08cf983b4c60 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:33.538567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:33.578211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:33.578859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.579136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.579397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] PATH_INFO: `/v3/users/77cc1591f764458f892fe2653fb68e14` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.580692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7467-6fe4-48f1-b1ed-f60da0e0a508 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:33.595358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] RBAC: Authorizing `identity:get_user(user_id=77cc1591f764458f892fe2653fb68e14)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:33.596784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:33.598480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 517/1058] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:33 2022] GET /identity/v3/users/77cc1591f764458f892fe2653fb68e14 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:33.604068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:33.618634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:33.619233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.619574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.619834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.690292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:33.691783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:33.720259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c31608-11e0-4768-ae8b-0ea02242ad3f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:33.758209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6426859-bd4e-4d46-bee3-968d27cd8b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:33.758849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426859-bd4e-4d46-bee3-968d27cd8b41 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.759076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426859-bd4e-4d46-bee3-968d27cd8b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.759350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426859-bd4e-4d46-bee3-968d27cd8b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.764170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:33.777072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:33.804869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 542/1059] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 14:45:33.810382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:33.835778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6426859-bd4e-4d46-bee3-968d27cd8b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:33.837251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6426859-bd4e-4d46-bee3-968d27cd8b41 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:33.907170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f7cc-9774-4556-ad19-9def0dfe63db None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:33.935283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 518/1060] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 14:45:33.936544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b498d-7408-4603-bfde-b9df7e0166a5 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.936829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b498d-7408-4603-bfde-b9df7e0166a5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.937105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b498d-7408-4603-bfde-b9df7e0166a5 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.938384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 519/1061] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:33.941124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b498d-7408-4603-bfde-b9df7e0166a5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:33.942442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892172c9-411b-49e6-bc88-4590dd7c5226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:33.943803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892172c9-411b-49e6-bc88-4590dd7c5226 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.944093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892172c9-411b-49e6-bc88-4590dd7c5226 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.944359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892172c9-411b-49e6-bc88-4590dd7c5226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.954627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892172c9-411b-49e6-bc88-4590dd7c5226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:33.956077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892172c9-411b-49e6-bc88-4590dd7c5226 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-892172c9-411b-49e6-bc88-4590dd7c5226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:33.970718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:33.971727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:33.972621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:33.972621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:33.972621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:33.972621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 543/1062] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:33.973411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c40b-be70-41b4-86ea-e45a202a4ce3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:33.973687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c40b-be70-41b4-86ea-e45a202a4ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:33.973951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c40b-be70-41b4-86ea-e45a202a4ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:33.991578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6815c40b-be70-41b4-86ea-e45a202a4ce3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:33.992748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6815c40b-be70-41b4-86ea-e45a202a4ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:33.996635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:34.003411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:34.056091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b498d-7408-4603-bfde-b9df7e0166a5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:34.059651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 544/1063] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:34.060661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815c40b-be70-41b4-86ea-e45a202a4ce3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:34.098719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:34.099564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:34.099989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:34.100417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:34.101564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c84ccef96e4d7a868db804cbe016de)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:34.103049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:34.105017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 520/1064] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:45:33 2022] GET /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:34.110672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:34.168224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815c40b-be70-41b4-86ea-e45a202a4ce3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:34.203360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:34.203979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:34.204282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:34.204530 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:34.205170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:34.205510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:34.206876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:34.213001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132a44e-6216-4593-b49f-951eaf9d0875 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:34.229232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 545/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:34.238549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:34.263371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:34.263994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:34.264273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:34.264542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:34.265320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=a0c84ccef96e4d7a868db804cbe016de, user.id=77cc1591f764458f892fe2653fb68e14) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:34.265626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:34.267054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:34.273380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 521/1066] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:34 2022] GET /identity/v3/role_assignments?user.id=77cc1591f764458f892fe2653fb68e14&scope.project.id=a0c84ccef96e4d7a868db804cbe016de&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:34.354328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:34.364437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:34.468747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ece97-eb20-4dbc-9480-2976853749df None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:34.503865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:34.504594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:34.504874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:34.505145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:34.518873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:34.524472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:34.525601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:34.526665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 546/1067] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 14:45:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:34.532673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:34.640703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc069de-1a31-4060-aac5-5f891042b9bf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:34.675624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:34.676222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:34.676496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:34.676761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:34.677457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:34.677756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:34.679097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:34.686826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 522/1068] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 14:45:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:34.692567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:34.756010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:34.791481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797bbe0-1f01-4ba9-8847-bfffd8269676 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:34.823248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:34.823870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:34.824155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:34.824427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:34.835133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:34.836572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:34.850689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:34.851684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:34.852557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:34.852557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:34.852557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:34.852557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 547/1069] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:34.857513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:34.898772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:34.954404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856c6a-f8a1-4aed-9858-5a99bc8cac73 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:34.987575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:34.988214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:34.988493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:34.988767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:34.989451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:34.989757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:34.991233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:35.007303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 523/1070] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:35.014018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:35.113064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556c3f8-e5ad-46a7-9a14-07833e33d251 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:35.129070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:35.147371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:35.148020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:35.148295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:35.148564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:35.157191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:35.158613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:35.170275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:35.171365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:35.172316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 548/1071] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:45:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:35.177221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:35.271176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:35.274109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939425-c8b1-440b-9a7e-df70a0398bc4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:35.306682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:35.307274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:35.307550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:35.307821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:35.308530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:35.308831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:35.310277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:35.331210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 524/1072] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:45:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:35.337026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:35.442314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58122db9-2d4e-47b2-af59-a42d821ba60b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:35.477145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:35.477795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:35.478073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:35.478344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:35.489071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:35.491073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:35.505041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:35.506647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:35.507526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:35.507526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:35.507526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:35.507526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 549/1073] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:35.512770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:35.607665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae8bec-d8f4-4be6-baad-6774a6744d2f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:35.642199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:35.642813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:35.643094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:35.643373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:35.644022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:35.644325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:35.645758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:35.657522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 525/1074] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:35.663834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:35.694563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:35.760922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125bf35-25d7-4144-8bd4-eb0e2fda5c68 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:35.793365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:35.793974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:35.794244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:35.794514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:35.800965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:35.802475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:35.813911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:35.814932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 550/1075] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:35.821061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:35.839488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:35.917579 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602c399-359c-4363-aa23-075275e2b162 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:35.949686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccc04d-d413-4851-b3f2-321c4e47e035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:35.950215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccc04d-d413-4851-b3f2-321c4e47e035 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:35.950488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccc04d-d413-4851-b3f2-321c4e47e035 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:35.950757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccc04d-d413-4851-b3f2-321c4e47e035 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:35.951427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccc04d-d413-4851-b3f2-321c4e47e035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:35.951724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccc04d-d413-4851-b3f2-321c4e47e035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:35.953116 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccc04d-d413-4851-b3f2-321c4e47e035 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:35.964672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 526/1076] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:35.971000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:36.071147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:36.081083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdccd61-91bd-4968-903a-366fe47ba895 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:36.114149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1465e-e969-48f4-93e5-2bc332d470cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:36.114798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1465e-e969-48f4-93e5-2bc332d470cb None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:36.115115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1465e-e969-48f4-93e5-2bc332d470cb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:36.115387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1465e-e969-48f4-93e5-2bc332d470cb None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/7b6809bf91d244e4bed5b8a256a3f95a/roles/e5c78c8957b941eaba90da16737d4525` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:36.117519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1465e-e969-48f4-93e5-2bc332d470cb None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=7b6809bf91d244e4bed5b8a256a3f95a, role_id=e5c78c8957b941eaba90da16737d4525)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:36.118946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1465e-e969-48f4-93e5-2bc332d470cb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:36.131147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 551/1077] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:45:35 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/7b6809bf91d244e4bed5b8a256a3f95a/roles/e5c78c8957b941eaba90da16737d4525 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:45:36.417489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27faaf-efd5-47b2-8fe6-c2a82c8592be None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:36.417773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27faaf-efd5-47b2-8fe6-c2a82c8592be None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:36.418040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27faaf-efd5-47b2-8fe6-c2a82c8592be None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:36.419389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 527/1078] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:36.424640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b63fa-5bf7-4b2a-9c8a-4b417c5d44e0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:36.424926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b63fa-5bf7-4b2a-9c8a-4b417c5d44e0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:36.425224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b63fa-5bf7-4b2a-9c8a-4b417c5d44e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:36.443526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ab7b63fa-5bf7-4b2a-9c8a-4b417c5d44e0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:36.444708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ab7b63fa-5bf7-4b2a-9c8a-4b417c5d44e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:36.510037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 552/1079] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:36.773498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27faaf-efd5-47b2-8fe6-c2a82c8592be None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:36.849249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:36.871893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27faaf-efd5-47b2-8fe6-c2a82c8592be None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:36.904973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:36.905682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:36.905961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:36.906229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:36.916597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:36.921366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:36.925723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:36.926782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 528/1080] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:45:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:36.928914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b63fa-5bf7-4b2a-9c8a-4b417c5d44e0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:37.010839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:37.034470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b63fa-5bf7-4b2a-9c8a-4b417c5d44e0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:37.066875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:37.067465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:37.067740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:37.068030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:37.068676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:37.068970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:37.070369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:37.077978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 553/1081] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:37.084378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:37.179672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59caf90-07ed-4e59-918c-19573c83ec65 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:37.212357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:37.212974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:37.213285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:37.213558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] PATH_INFO: `/v3/users/77cc1591f764458f892fe2653fb68e14` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:37.214671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] RBAC: Authorizing `identity:get_user(user_id=77cc1591f764458f892fe2653fb68e14)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:37.216082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:37.217692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 529/1082] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:37 2022] GET /identity/v3/users/77cc1591f764458f892fe2653fb68e14 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:37.223255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:37.318917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810eba6-6c4f-4f04-af2b-d80df0a2e268 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:37.353122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8622e7f-c2b1-4624-94f4-e40c80b878eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:37.353771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622e7f-c2b1-4624-94f4-e40c80b878eb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:37.354050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622e7f-c2b1-4624-94f4-e40c80b878eb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:37.354323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622e7f-c2b1-4624-94f4-e40c80b878eb None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:37.355422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8622e7f-c2b1-4624-94f4-e40c80b878eb None admin] RBAC: Authorizing `identity:get_project(project_id=a0c84ccef96e4d7a868db804cbe016de)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:37.356861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8622e7f-c2b1-4624-94f4-e40c80b878eb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:37.358525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 554/1083] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:45:37 2022] GET /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:37.364409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:37.373384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:37.461723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff164f-64a3-4145-8b26-12ad78c0cd67 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:37.496874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cb99b-283e-4be0-851d-cfa4907458aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:37.497553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb99b-283e-4be0-851d-cfa4907458aa None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:37.497827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb99b-283e-4be0-851d-cfa4907458aa None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:37.498100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb99b-283e-4be0-851d-cfa4907458aa None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de/users/77cc1591f764458f892fe2653fb68e14/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:37.499955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cb99b-283e-4be0-851d-cfa4907458aa None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c84ccef96e4d7a868db804cbe016de, user_id=77cc1591f764458f892fe2653fb68e14, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:37.501389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cb99b-283e-4be0-851d-cfa4907458aa None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:37.514385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 530/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:45:37 2022] PUT /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de/users/77cc1591f764458f892fe2653fb68e14/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:45:38.249657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879acd6d-dd38-407e-aa5b-2fb045440e28 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:38.250195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879acd6d-dd38-407e-aa5b-2fb045440e28 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:38.250195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879acd6d-dd38-407e-aa5b-2fb045440e28 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:38.251658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 555/1085] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:38.256616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2104e6-d615-4b17-b1df-11e45dd8ac50 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:38.257053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2104e6-d615-4b17-b1df-11e45dd8ac50 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:38.257154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2104e6-d615-4b17-b1df-11e45dd8ac50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:38.275727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9c2104e6-d615-4b17-b1df-11e45dd8ac50 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:38.276821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9c2104e6-d615-4b17-b1df-11e45dd8ac50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:38.335889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:38.349340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 531/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:38.617298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879acd6d-dd38-407e-aa5b-2fb045440e28 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:38.629888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:38.726111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879acd6d-dd38-407e-aa5b-2fb045440e28 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:38.759230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:38.759850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:38.760128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:38.760397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:38.770442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:38.775237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:38.776397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:38.777354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 556/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 14:45:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:38.782229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2104e6-d615-4b17-b1df-11e45dd8ac50 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:38.876684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2104e6-d615-4b17-b1df-11e45dd8ac50 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:38.908578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:38.909171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:38.909476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:38.909745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:38.910394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:38.910692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:38.912558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:38.924717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 532/1088] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 14:45:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:38.925856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:38.993997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:39.027473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad237e45-9e76-4219-bd3e-2fc960322839 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:39.061092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:39.061740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:39.062021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:39.062292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:39.071995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:39.073457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:39.088017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:39.094065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:39.095003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:39.095003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:39.095003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:39.095003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 557/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:39.095249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:39.152284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:39.192790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a871e-6052-4ffb-a83b-8e59e322ea92 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:39.225648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:39.226249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:39.226527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:39.226798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:39.227427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:39.227732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:39.229086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:39.240500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 533/1090] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:39.246273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:39.343178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c7b3-6dc4-419e-a8d0-09a58b888482 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:39.364958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:39.376658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:39.377310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:39.377588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:39.377857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:39.385558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:39.387247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:45:39.398634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:39.399776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:39.400727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 558/1091] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 14:45:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:39.405671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:39.507196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc91dc2-71da-475a-bd60-cb7c72e20200 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:39.526982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:39.544262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595717f0-23f0-4724-a257-758c2857995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:39.544870 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595717f0-23f0-4724-a257-758c2857995a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:39.545149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595717f0-23f0-4724-a257-758c2857995a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:39.545445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595717f0-23f0-4724-a257-758c2857995a None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:39.546165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595717f0-23f0-4724-a257-758c2857995a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:39.546466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595717f0-23f0-4724-a257-758c2857995a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:39.547872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595717f0-23f0-4724-a257-758c2857995a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:39.564675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 534/1092] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 14:45:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:39.571370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:39.620657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa6df-2daa-4aea-9ef9-c27bcd6fc058 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:39.620941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa6df-2daa-4aea-9ef9-c27bcd6fc058 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:39.621250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa6df-2daa-4aea-9ef9-c27bcd6fc058 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:39.622599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 535/1093] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:45:39.627763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b22844-7c17-40ee-825f-b957d9c29175 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:39.628046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b22844-7c17-40ee-825f-b957d9c29175 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:39.628322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b22844-7c17-40ee-825f-b957d9c29175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:39.647300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-61b22844-7c17-40ee-825f-b957d9c29175 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:39.648488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-61b22844-7c17-40ee-825f-b957d9c29175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:39.671179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197801-314b-4dea-9cfe-13b6f7e0ffb9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:39.703596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:39.704195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:39.704470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:39.704737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:39.713210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 536/1094] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:39.751652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:39.770437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:39.771918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:39.869928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 559/1095] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:45:39.871285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b22844-7c17-40ee-825f-b957d9c29175 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:39.969755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b22844-7c17-40ee-825f-b957d9c29175 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:39.977830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:39.986243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.006196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.006824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.007101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.007370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.020590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.022063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:45:40.036319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:40.037517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:45:40.038648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:45:40.038648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:45:40.038648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:40.038648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 537/1096] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:45:40.042902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.075036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:40.092652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfabd5-73a2-4efb-bc9b-697f7c5ff9c7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.125779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.126401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.126683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.126965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.137794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:45:40.141704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9818a-5e53-45fa-99e0-82714eecd5fc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:45:40.142603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:40.143518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:45:40.144485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 560/1097] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:40.149592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.175167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.175769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.176055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.176328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.176962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:40.177299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.179251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.190705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 538/1098] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:40.196189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.213985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:40.251832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf358b44-8266-4648-ad55-70368d5bba35 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.285727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.286318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.286592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.287347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.287989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:40.288282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.289661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.297512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 561/1099] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:40.297656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce37d5-2e1e-4360-89bb-51912dfe1de0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.303730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.307349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:40.332805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.333482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.333904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.334178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.340893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.342363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:45:40.353958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:45:40.355143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:45:40.356065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:45:40.357530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 539/1100] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:45:40.360657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.402980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b474-d197-4c61-bc24-97870d811432 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.434997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:40.437533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.438164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.438449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.438722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] PATH_INFO: `/v3/users/77cc1591f764458f892fe2653fb68e14` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.439852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] RBAC: Authorizing `identity:get_user(user_id=77cc1591f764458f892fe2653fb68e14)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.441349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.442930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 562/1101] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/users/77cc1591f764458f892fe2653fb68e14 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:40.448690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.459491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b9bc-d636-432f-a239-0436869a6d92 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.493859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.494467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.494737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.495004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.495683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:40.495983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.497418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.509413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 540/1102] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:45:40.515579 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.525097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:40.555571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f40ed-719b-4426-8aca-ec6fb2902a08 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.588419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.589020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.589335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.589611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.616681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aedc4-daf6-4caa-b56c-5615b2ef0702 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.651370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.651954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.652229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.652492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.653210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] RBAC: Adding query filter params (role.id=e5c78c8957b941eaba90da16737d4525, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=7b6809bf91d244e4bed5b8a256a3f95a) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:40.653526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.654918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.655304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.656747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.660576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 541/1103] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/role_assignments?user.id=7b6809bf91d244e4bed5b8a256a3f95a&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=e5c78c8957b941eaba90da16737d4525 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:40.730063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:40.762591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 563/1104] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 14:45:40.768682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:40.771681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:40.871611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed9ddf-d465-4451-bbea-9e7221ae6784 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:40.906847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7fd4c-c602-42dc-aa61-f80fc2552b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:40.907472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7fd4c-c602-42dc-aa61-f80fc2552b3b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:40.907748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7fd4c-c602-42dc-aa61-f80fc2552b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:40.908022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7fd4c-c602-42dc-aa61-f80fc2552b3b None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:40.909116 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7fd4c-c602-42dc-aa61-f80fc2552b3b None admin] RBAC: Authorizing `identity:get_project(project_id=a0c84ccef96e4d7a868db804cbe016de)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:40.910563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7fd4c-c602-42dc-aa61-f80fc2552b3b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:40.912962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 542/1105] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:40.918922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:41.020574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64701a47-0604-48f8-95a7-b55342e95cd9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:41.055600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778073fe-a54a-41af-912a-c03304ec013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:41.056201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778073fe-a54a-41af-912a-c03304ec013d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:41.056479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778073fe-a54a-41af-912a-c03304ec013d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:41.056753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778073fe-a54a-41af-912a-c03304ec013d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:41.057507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778073fe-a54a-41af-912a-c03304ec013d None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=a0c84ccef96e4d7a868db804cbe016de, user.id=77cc1591f764458f892fe2653fb68e14) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:41.057812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778073fe-a54a-41af-912a-c03304ec013d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:41.059246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778073fe-a54a-41af-912a-c03304ec013d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:41.065063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 564/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:40 2022] GET /identity/v3/role_assignments?user.id=77cc1591f764458f892fe2653fb68e14&scope.project.id=a0c84ccef96e4d7a868db804cbe016de&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:43.235684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e03cc-3171-49c9-ae5b-6537740f2a58 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:43.236218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e03cc-3171-49c9-ae5b-6537740f2a58 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:43.236218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e03cc-3171-49c9-ae5b-6537740f2a58 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:43.237602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 543/1107] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:43.243601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e8c9e-b074-4844-813b-643cbca144a7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:43.244021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e8c9e-b074-4844-813b-643cbca144a7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:43.244196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e8c9e-b074-4844-813b-643cbca144a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:43.261979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6e4e8c9e-b074-4844-813b-643cbca144a7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:43.263141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6e4e8c9e-b074-4844-813b-643cbca144a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:43.290707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:43.327440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 565/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:43.601231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e03cc-3171-49c9-ae5b-6537740f2a58 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:43.668942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:43.699205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e03cc-3171-49c9-ae5b-6537740f2a58 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:43.732173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e58c4c-7c2c-4bf7-a5b9-54ec0b97e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:43.732786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e58c4c-7c2c-4bf7-a5b9-54ec0b97e68f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:43.733067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e58c4c-7c2c-4bf7-a5b9-54ec0b97e68f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:43.733368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e58c4c-7c2c-4bf7-a5b9-54ec0b97e68f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:43.734475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e58c4c-7c2c-4bf7-a5b9-54ec0b97e68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:43.735889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e58c4c-7c2c-4bf7-a5b9-54ec0b97e68f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:43.737464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 544/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:43.743112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e8c9e-b074-4844-813b-643cbca144a7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:43.836479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e8c9e-b074-4844-813b-643cbca144a7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:43.868684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52525de9-e285-4d90-bce4-5e69a7f3669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:43.869232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52525de9-e285-4d90-bce4-5e69a7f3669b None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:43.869510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52525de9-e285-4d90-bce4-5e69a7f3669b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:43.869773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52525de9-e285-4d90-bce4-5e69a7f3669b None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:43.870402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52525de9-e285-4d90-bce4-5e69a7f3669b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:43.871807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52525de9-e285-4d90-bce4-5e69a7f3669b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:43.892438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 566/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:45:43 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:46.026692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df219-bedb-473d-bd89-8432940cae51 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:46.027116 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df219-bedb-473d-bd89-8432940cae51 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:46.027237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df219-bedb-473d-bd89-8432940cae51 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:46.028574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 545/1111] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:46.033675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980655b-ef2b-49ad-86f5-916034b3de65 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:46.034093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980655b-ef2b-49ad-86f5-916034b3de65 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:46.034225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980655b-ef2b-49ad-86f5-916034b3de65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:46.052565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9980655b-ef2b-49ad-86f5-916034b3de65 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:46.053790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9980655b-ef2b-49ad-86f5-916034b3de65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:46.069583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:46.119379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 567/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:46.385946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df219-bedb-473d-bd89-8432940cae51 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:46.484192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df219-bedb-473d-bd89-8432940cae51 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:46.517575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:46.518191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:46.518473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:46.518749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] PATH_INFO: `/v3/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:46.520127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] RBAC: Authorizing `identity:get_role(role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:46.521506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:46.522820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 546/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:46 2022] GET /identity/v3/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:46.528594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980655b-ef2b-49ad-86f5-916034b3de65 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:46.623035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980655b-ef2b-49ad-86f5-916034b3de65 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:46.655808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:46.656357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:46.656631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:46.656899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] PATH_INFO: `/v3/users/22aa328b3b7247bb80130ca8c2ca6983` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:46.669341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] RBAC: Authorizing `identity:get_user(user_id=22aa328b3b7247bb80130ca8c2ca6983)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:46.670754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:46.672299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 568/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:46 2022] GET /identity/v3/users/22aa328b3b7247bb80130ca8c2ca6983 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:46.678514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:46.687107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:46.774773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcb486-d3c2-40d4-86a5-3430a5da8d00 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:46.806231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:46.806817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:46.807093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:46.807363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:46.870077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:46.871482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:46.878642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:46.964565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 547/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:45:46.970527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:46.982801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:47.067123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70188e3-5ef7-418b-a88a-8f5370f28f73 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:47.098671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d3618-4c24-4817-a4e2-8a94257855d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:47.099810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d3618-4c24-4817-a4e2-8a94257855d7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:47.100084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d3618-4c24-4817-a4e2-8a94257855d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:47.100351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d3618-4c24-4817-a4e2-8a94257855d7 None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:47.101470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d3618-4c24-4817-a4e2-8a94257855d7 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c84ccef96e4d7a868db804cbe016de)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:47.102890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d3618-4c24-4817-a4e2-8a94257855d7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:47.104489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 569/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:45:46 2022] GET /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:47.110570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:47.207619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ae948-4891-44a1-b14f-73d77eb119e1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:47.214689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:47.243694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f7135-8c4e-4034-afe5-d23be43e8354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:47.244293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f7135-8c4e-4034-afe5-d23be43e8354 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:47.244569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f7135-8c4e-4034-afe5-d23be43e8354 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:47.244925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f7135-8c4e-4034-afe5-d23be43e8354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:47.245678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f7135-8c4e-4034-afe5-d23be43e8354 None admin] RBAC: Adding query filter params (role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=a0c84ccef96e4d7a868db804cbe016de, user.id=22aa328b3b7247bb80130ca8c2ca6983) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:47.245985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f7135-8c4e-4034-afe5-d23be43e8354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:47.247390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f7135-8c4e-4034-afe5-d23be43e8354 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:47.252646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 548/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:47 2022] GET /identity/v3/role_assignments?user.id=22aa328b3b7247bb80130ca8c2ca6983&scope.project.id=a0c84ccef96e4d7a868db804cbe016de&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:49.346795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4f7a-b7b0-4aa5-a77a-0a0987717da6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:49.347312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4f7a-b7b0-4aa5-a77a-0a0987717da6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:49.347312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4f7a-b7b0-4aa5-a77a-0a0987717da6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:49.348568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 570/1118] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:49.353767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf243a7c-d0d9-45f5-8d58-2e52e53e3ef7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:49.354187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf243a7c-d0d9-45f5-8d58-2e52e53e3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:49.354303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf243a7c-d0d9-45f5-8d58-2e52e53e3ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:49.371965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cf243a7c-d0d9-45f5-8d58-2e52e53e3ef7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:49.373132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cf243a7c-d0d9-45f5-8d58-2e52e53e3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:49.436965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 549/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:49.703377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4f7a-b7b0-4aa5-a77a-0a0987717da6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:49.779001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:49.801375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4f7a-b7b0-4aa5-a77a-0a0987717da6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:49.834328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:49.834944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:49.835219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:49.835486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] PATH_INFO: `/v3/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:49.836848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] RBAC: Authorizing `identity:get_role(role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:49.838237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:49.839533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 571/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:49 2022] GET /identity/v3/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:49.845475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf243a7c-d0d9-45f5-8d58-2e52e53e3ef7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:49.936983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:49.939856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf243a7c-d0d9-45f5-8d58-2e52e53e3ef7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:49.971984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:49.972536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:49.972810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:49.973076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] PATH_INFO: `/v3/users/22aa328b3b7247bb80130ca8c2ca6983` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:49.974694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] RBAC: Authorizing `identity:get_user(user_id=22aa328b3b7247bb80130ca8c2ca6983)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:49.976098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:49.977682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 550/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:49 2022] GET /identity/v3/users/22aa328b3b7247bb80130ca8c2ca6983 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:49.983346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:50.085574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bf7f8-b339-40cf-a7ed-003c66a820ef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:50.118239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2f894-9aca-4d97-8fc0-a91c3c04101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:50.118848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f894-9aca-4d97-8fc0-a91c3c04101a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:50.119124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f894-9aca-4d97-8fc0-a91c3c04101a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:50.119391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f894-9aca-4d97-8fc0-a91c3c04101a None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:50.120476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2f894-9aca-4d97-8fc0-a91c3c04101a None admin] RBAC: Authorizing `identity:get_project(project_id=a0c84ccef96e4d7a868db804cbe016de)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:50.121925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2f894-9aca-4d97-8fc0-a91c3c04101a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:50.123512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 572/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:45:49 2022] GET /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:50.129643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:50.224853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07899468-80fb-4d49-8a31-c0dc0f88a51f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:50.257243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d8482-2803-4c9d-b6b3-fdaa05016b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:50.257890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8482-2803-4c9d-b6b3-fdaa05016b46 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:50.258165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8482-2803-4c9d-b6b3-fdaa05016b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:50.258435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8482-2803-4c9d-b6b3-fdaa05016b46 None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de/users/22aa328b3b7247bb80130ca8c2ca6983/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:50.260521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d8482-2803-4c9d-b6b3-fdaa05016b46 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c84ccef96e4d7a868db804cbe016de, user_id=22aa328b3b7247bb80130ca8c2ca6983, role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:50.262204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d8482-2803-4c9d-b6b3-fdaa05016b46 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:50.274823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 551/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:45:50 2022] PUT /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de/users/22aa328b3b7247bb80130ca8c2ca6983/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:45:52.358204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67368-2f6b-4f2d-ac90-3c0d3019edde None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:52.358728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67368-2f6b-4f2d-ac90-3c0d3019edde None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:52.358728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67368-2f6b-4f2d-ac90-3c0d3019edde None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:52.359983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 573/1124] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:52.365104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5dc9a-c3e6-412d-ae13-6a16b4c6cb4d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:52.365499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5dc9a-c3e6-412d-ae13-6a16b4c6cb4d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:52.365667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5dc9a-c3e6-412d-ae13-6a16b4c6cb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:52.383714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2ca5dc9a-c3e6-412d-ae13-6a16b4c6cb4d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:52.384873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2ca5dc9a-c3e6-412d-ae13-6a16b4c6cb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:52.407199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:52.449245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 552/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:52.713053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67368-2f6b-4f2d-ac90-3c0d3019edde None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:52.728690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:52.812400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67368-2f6b-4f2d-ac90-3c0d3019edde None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:52.844620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:52.845252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:52.845533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:52.845805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] PATH_INFO: `/v3/roles/aa00fbf53b9f42adbca10b4eb2405a96` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:52.847181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] RBAC: Authorizing `identity:get_role(role_id=aa00fbf53b9f42adbca10b4eb2405a96)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:52.848540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:52.849885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 574/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:52 2022] GET /identity/v3/roles/aa00fbf53b9f42adbca10b4eb2405a96 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:52.855556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5dc9a-c3e6-412d-ae13-6a16b4c6cb4d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:52.948984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5dc9a-c3e6-412d-ae13-6a16b4c6cb4d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:52.980933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:52.981513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:52.981788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:52.982055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] PATH_INFO: `/v3/users/22aa328b3b7247bb80130ca8c2ca6983` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:52.983150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] RBAC: Authorizing `identity:get_user(user_id=22aa328b3b7247bb80130ca8c2ca6983)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:52.984554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:52.986139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 553/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:45:52 2022] GET /identity/v3/users/22aa328b3b7247bb80130ca8c2ca6983 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:52.992503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:53.071818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:53.089693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b959ae-4fd1-409f-9500-addd59a324b0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:53.122675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:53.123276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:53.123559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:53.123834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:53.188427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:53.189897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:53.265854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:53.284657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 575/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:45:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:45:53.290724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:53.317117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:53.386521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f0cac-749d-4b4e-9633-13884e6e5531 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:53.419391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf06c7a-583b-4f4c-a3a0-7866978bedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:53.420001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf06c7a-583b-4f4c-a3a0-7866978bedf9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:53.420277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf06c7a-583b-4f4c-a3a0-7866978bedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:53.420545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf06c7a-583b-4f4c-a3a0-7866978bedf9 None admin] PATH_INFO: `/v3/projects/a0c84ccef96e4d7a868db804cbe016de` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:53.421670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf06c7a-583b-4f4c-a3a0-7866978bedf9 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c84ccef96e4d7a868db804cbe016de)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:53.423087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf06c7a-583b-4f4c-a3a0-7866978bedf9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:53.424673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 554/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:45:53 2022] GET /identity/v3/projects/a0c84ccef96e4d7a868db804cbe016de => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:53.430702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:53.524408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a3100-2dc0-44a1-81dc-ac8a4a574e98 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:53.556672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878908d-8358-4c9f-a4b7-61aa4284debe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:53.557276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878908d-8358-4c9f-a4b7-61aa4284debe None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:53.557554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878908d-8358-4c9f-a4b7-61aa4284debe None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:53.557822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878908d-8358-4c9f-a4b7-61aa4284debe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:53.558525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878908d-8358-4c9f-a4b7-61aa4284debe None admin] RBAC: Adding query filter params (role.id=aa00fbf53b9f42adbca10b4eb2405a96, scope.project.id=a0c84ccef96e4d7a868db804cbe016de, user.id=22aa328b3b7247bb80130ca8c2ca6983) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:45:53.558822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878908d-8358-4c9f-a4b7-61aa4284debe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:53.560228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878908d-8358-4c9f-a4b7-61aa4284debe None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:53.565743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 576/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:45:53 2022] GET /identity/v3/role_assignments?user.id=22aa328b3b7247bb80130ca8c2ca6983&scope.project.id=a0c84ccef96e4d7a868db804cbe016de&role.id=aa00fbf53b9f42adbca10b4eb2405a96 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:55.693090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482b5fb-07ad-4777-b89a-b3825d0012e8 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:55.693640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482b5fb-07ad-4777-b89a-b3825d0012e8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:55.693640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482b5fb-07ad-4777-b89a-b3825d0012e8 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:55.695028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 555/1131] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:55.700288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6cfca-e168-4110-9886-89b64ad63129 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:55.700712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6cfca-e168-4110-9886-89b64ad63129 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:55.700818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6cfca-e168-4110-9886-89b64ad63129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:55.718361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-45e6cfca-e168-4110-9886-89b64ad63129 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:55.719524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-45e6cfca-e168-4110-9886-89b64ad63129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:55.760309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:55.784389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 577/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:56.050795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482b5fb-07ad-4777-b89a-b3825d0012e8 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:56.141985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:56.149776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482b5fb-07ad-4777-b89a-b3825d0012e8 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:56.183000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef9459-7db4-408e-9e94-ee3b94c7a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:56.183612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9459-7db4-408e-9e94-ee3b94c7a007 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:56.183889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9459-7db4-408e-9e94-ee3b94c7a007 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:56.184158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9459-7db4-408e-9e94-ee3b94c7a007 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:56.185259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef9459-7db4-408e-9e94-ee3b94c7a007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:56.186666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef9459-7db4-408e-9e94-ee3b94c7a007 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:56.188206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 556/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:56.193994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6cfca-e168-4110-9886-89b64ad63129 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:56.288813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6cfca-e168-4110-9886-89b64ad63129 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:56.320800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b2edf-ace7-4bc5-a5bb-ab46ca73710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:56.321416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2edf-ace7-4bc5-a5bb-ab46ca73710f None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:56.321701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2edf-ace7-4bc5-a5bb-ab46ca73710f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:56.321972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2edf-ace7-4bc5-a5bb-ab46ca73710f None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:56.322616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b2edf-ace7-4bc5-a5bb-ab46ca73710f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:56.324037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b2edf-ace7-4bc5-a5bb-ab46ca73710f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:56.350050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 578/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:45:56 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:45:58.473808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad9b45-af79-404a-8a86-97eecb8e3c01 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:58.474334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad9b45-af79-404a-8a86-97eecb8e3c01 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:58.474334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad9b45-af79-404a-8a86-97eecb8e3c01 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:58.475714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 557/1135] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:45:58.480821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d56cc8-97c0-414b-bd95-20fdb4b85e6b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:58.481238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d56cc8-97c0-414b-bd95-20fdb4b85e6b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:58.481387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d56cc8-97c0-414b-bd95-20fdb4b85e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:58.499509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-67d56cc8-97c0-414b-bd95-20fdb4b85e6b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:45:58.500675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-67d56cc8-97c0-414b-bd95-20fdb4b85e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:45:58.523093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:45:58.566440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 579/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:45:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:45:58.832029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad9b45-af79-404a-8a86-97eecb8e3c01 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:58.929711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad9b45-af79-404a-8a86-97eecb8e3c01 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:58.962612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95683e4d-0959-44fd-b603-66ac55f9a406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:58.963224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683e4d-0959-44fd-b603-66ac55f9a406 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:58.963503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683e4d-0959-44fd-b603-66ac55f9a406 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:58.963772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683e4d-0959-44fd-b603-66ac55f9a406 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:58.964869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95683e4d-0959-44fd-b603-66ac55f9a406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:58.966320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95683e4d-0959-44fd-b603-66ac55f9a406 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:58.967849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 558/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:45:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:45:58.973537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d56cc8-97c0-414b-bd95-20fdb4b85e6b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:45:59.067189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d56cc8-97c0-414b-bd95-20fdb4b85e6b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:45:59.099902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed57b3d-b411-40fc-8d93-4a66bde73ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:45:59.100435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed57b3d-b411-40fc-8d93-4a66bde73ce8 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:45:59.100713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed57b3d-b411-40fc-8d93-4a66bde73ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:45:59.100981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed57b3d-b411-40fc-8d93-4a66bde73ce8 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:45:59.101639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed57b3d-b411-40fc-8d93-4a66bde73ce8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:45:59.103056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed57b3d-b411-40fc-8d93-4a66bde73ce8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:45:59.123299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 580/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:45:58 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:46:01.285662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76e8d4-d6fa-4028-bed5-ea4eb3eb0ad1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:01.286774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76e8d4-d6fa-4028-bed5-ea4eb3eb0ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:01.286774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76e8d4-d6fa-4028-bed5-ea4eb3eb0ad1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:01.288180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 559/1139] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:01.294043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3e7f0-64a6-48bd-9d5e-f7f390a11d0f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:01.294502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3e7f0-64a6-48bd-9d5e-f7f390a11d0f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:01.294603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3e7f0-64a6-48bd-9d5e-f7f390a11d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:01.313836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-77f3e7f0-64a6-48bd-9d5e-f7f390a11d0f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:01.315024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-77f3e7f0-64a6-48bd-9d5e-f7f390a11d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:01.324023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:01.379807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 581/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:01.668158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76e8d4-d6fa-4028-bed5-ea4eb3eb0ad1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:01.698190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:01.771470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76e8d4-d6fa-4028-bed5-ea4eb3eb0ad1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:01.806056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:01.806677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:01.806957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:01.807230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:01.817044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:46:01.821881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:46:01.823016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:46:01.823961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 560/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:46:01.829295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3e7f0-64a6-48bd-9d5e-f7f390a11d0f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:01.925463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3e7f0-64a6-48bd-9d5e-f7f390a11d0f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:01.958223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:01.958821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:01.959096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:01.959368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:01.960024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:01.960320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:01.961725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:01.969473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 582/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:01.976001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:02.065101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:02.074714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e81cf-2d58-4915-bedb-244af32a9584 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:02.107145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:02.107772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:02.108033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:02.108301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] PATH_INFO: `/v3/users/482d3d075fc34691a5f6408fc77954d7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:02.121092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] RBAC: Authorizing `identity:get_user(user_id=482d3d075fc34691a5f6408fc77954d7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:02.122538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:02.124119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 561/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:46:01 2022] GET /identity/v3/users/482d3d075fc34691a5f6408fc77954d7 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:02.129909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:02.139214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:02.226300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2630c9-120b-4dcc-b202-11556a9d32f4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:02.258290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:02.258882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:02.259160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:02.259432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:02.323031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:02.324473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:02.331771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:02.419205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 583/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:46:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:46:02.425431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:02.519643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c3922-5b6a-4ac0-a877-720af163a205 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:02.551972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d969-fcff-4adf-b3c1-b1b5b3e84c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:02.552576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d969-fcff-4adf-b3c1-b1b5b3e84c27 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:02.552851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d969-fcff-4adf-b3c1-b1b5b3e84c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:02.553118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d969-fcff-4adf-b3c1-b1b5b3e84c27 None admin] PATH_INFO: `/v3/projects/4289c270585445a6aea511da436f75be` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:02.554236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06d969-fcff-4adf-b3c1-b1b5b3e84c27 None admin] RBAC: Authorizing `identity:get_project(project_id=4289c270585445a6aea511da436f75be)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:02.555642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06d969-fcff-4adf-b3c1-b1b5b3e84c27 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:02.557234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 562/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:46:02 2022] GET /identity/v3/projects/4289c270585445a6aea511da436f75be => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:02.563089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:02.657155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fbb64-69e2-406f-a094-edd4da28035b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:02.664152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:02.689710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfbe08-634a-40e0-8a3b-e3ee343581e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:02.690316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfbe08-634a-40e0-8a3b-e3ee343581e9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:02.690592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfbe08-634a-40e0-8a3b-e3ee343581e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:02.690860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfbe08-634a-40e0-8a3b-e3ee343581e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:02.691564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfbe08-634a-40e0-8a3b-e3ee343581e9 None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=4289c270585445a6aea511da436f75be, user.id=482d3d075fc34691a5f6408fc77954d7) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:02.691863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfbe08-634a-40e0-8a3b-e3ee343581e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:02.693866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfbe08-634a-40e0-8a3b-e3ee343581e9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:02.698982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 584/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:46:02 2022] GET /identity/v3/role_assignments?user.id=482d3d075fc34691a5f6408fc77954d7&scope.project.id=4289c270585445a6aea511da436f75be&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:04.771109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2edf14-2e81-41c6-ae6f-f6167543d0d7 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:04.771640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2edf14-2e81-41c6-ae6f-f6167543d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:04.771640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2edf14-2e81-41c6-ae6f-f6167543d0d7 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:04.772967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 563/1147] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:04.778368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ad732-12c4-4ddd-9b7f-b465cd16a515 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:04.778791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ad732-12c4-4ddd-9b7f-b465cd16a515 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:04.778913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ad732-12c4-4ddd-9b7f-b465cd16a515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:04.796448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bb3ad732-12c4-4ddd-9b7f-b465cd16a515 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:04.797646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bb3ad732-12c4-4ddd-9b7f-b465cd16a515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:04.860182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 585/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:05.126235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2edf14-2e81-41c6-ae6f-f6167543d0d7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:05.144055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:05.222056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2edf14-2e81-41c6-ae6f-f6167543d0d7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:05.254376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:05.255024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:05.255299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:05.255565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:05.265422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:46:05.270127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:46:05.271677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:46:05.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 564/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:46:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:46:05.277856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ad732-12c4-4ddd-9b7f-b465cd16a515 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:05.371941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:05.374873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ad732-12c4-4ddd-9b7f-b465cd16a515 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:05.408865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:05.409443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:05.409725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:05.409998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:05.410651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:05.411502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:05.412875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:05.420787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 586/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:46:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:05.426572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:05.504113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:05.527136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35d27-6f7b-43d8-b68b-45e1cea08cd1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:05.559586 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:05.560206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:05.560484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:05.560753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] PATH_INFO: `/v3/users/482d3d075fc34691a5f6408fc77954d7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:05.561871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] RBAC: Authorizing `identity:get_user(user_id=482d3d075fc34691a5f6408fc77954d7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:05.563290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:05.564866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 565/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:46:05 2022] GET /identity/v3/users/482d3d075fc34691a5f6408fc77954d7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:05.570686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:05.667504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc3e7e-dc6e-41f3-bc72-e1fc4e5982e4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:05.699540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371af47c-4072-4b27-989c-de35d78383b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:05.700151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371af47c-4072-4b27-989c-de35d78383b9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:05.700432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371af47c-4072-4b27-989c-de35d78383b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:05.700705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371af47c-4072-4b27-989c-de35d78383b9 None admin] PATH_INFO: `/v3/projects/4289c270585445a6aea511da436f75be` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:05.701806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371af47c-4072-4b27-989c-de35d78383b9 None admin] RBAC: Authorizing `identity:get_project(project_id=4289c270585445a6aea511da436f75be)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:05.703234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371af47c-4072-4b27-989c-de35d78383b9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:05.704816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 587/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:46:05 2022] GET /identity/v3/projects/4289c270585445a6aea511da436f75be => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:05.710459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:05.806278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6276f-a3a7-4f82-96f1-470616533a27 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:05.840377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7ebef-2479-4691-a8b6-e4a77f43fe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:05.841025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ebef-2479-4691-a8b6-e4a77f43fe76 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:05.841325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ebef-2479-4691-a8b6-e4a77f43fe76 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:05.841596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ebef-2479-4691-a8b6-e4a77f43fe76 None admin] PATH_INFO: `/v3/projects/4289c270585445a6aea511da436f75be/users/482d3d075fc34691a5f6408fc77954d7/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:05.843452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7ebef-2479-4691-a8b6-e4a77f43fe76 None admin] RBAC: Authorizing `identity:create_grant(project_id=4289c270585445a6aea511da436f75be, user_id=482d3d075fc34691a5f6408fc77954d7, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:05.844859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7ebef-2479-4691-a8b6-e4a77f43fe76 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:05.858355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 566/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:46:05 2022] PUT /identity/v3/projects/4289c270585445a6aea511da436f75be/users/482d3d075fc34691a5f6408fc77954d7/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:46:08.005967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93319c24-90d6-49ba-9bee-eec1b34de4ff None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:08.006543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93319c24-90d6-49ba-9bee-eec1b34de4ff None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:08.006655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93319c24-90d6-49ba-9bee-eec1b34de4ff None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:08.007925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 588/1154] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:08.013553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9064bf-d7f0-485a-a9bb-3d02673c746f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:08.013964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9064bf-d7f0-485a-a9bb-3d02673c746f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:08.014142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9064bf-d7f0-485a-a9bb-3d02673c746f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:08.021841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:08.033283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1d9064bf-d7f0-485a-a9bb-3d02673c746f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:08.034461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1d9064bf-d7f0-485a-a9bb-3d02673c746f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:08.097970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 567/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:08.366693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93319c24-90d6-49ba-9bee-eec1b34de4ff None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:08.397146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:08.467308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93319c24-90d6-49ba-9bee-eec1b34de4ff None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:08.501317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:08.501932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:08.502202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:08.502466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:08.513005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:46:08.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:46:08.518895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:46:08.519991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 589/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:46:08.524952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9064bf-d7f0-485a-a9bb-3d02673c746f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:08.619609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9064bf-d7f0-485a-a9bb-3d02673c746f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:08.652936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:08.653549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:08.653828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:08.654098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:08.654756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:08.655052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:08.656405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:08.657718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:08.665465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 568/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:46:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:08.671310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:08.757734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:08.767742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3a4b0-e721-4e5f-af13-8f4cc28c0ada None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:08.800090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:08.800690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:08.800965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:08.801256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] PATH_INFO: `/v3/users/482d3d075fc34691a5f6408fc77954d7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:08.802944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] RBAC: Authorizing `identity:get_user(user_id=482d3d075fc34691a5f6408fc77954d7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:08.804356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:08.805918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 590/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:46:08 2022] GET /identity/v3/users/482d3d075fc34691a5f6408fc77954d7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:08.811546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:08.905762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f459e-833a-4b1e-9d80-b3452ea5d483 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:08.938058 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:08.938643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:08.938924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:08.939193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:08.996611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:09.003417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:09.004860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:09.099876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 569/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:46:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:46:09.105992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:09.202715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac22894-01e5-4683-a5ec-10e77b337e9a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:09.235057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503278b-c7d1-4056-ab27-9f84b06415cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:09.235666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503278b-c7d1-4056-ab27-9f84b06415cc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:09.235939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503278b-c7d1-4056-ab27-9f84b06415cc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:09.236212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503278b-c7d1-4056-ab27-9f84b06415cc None admin] PATH_INFO: `/v3/projects/4289c270585445a6aea511da436f75be` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:09.237328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503278b-c7d1-4056-ab27-9f84b06415cc None admin] RBAC: Authorizing `identity:get_project(project_id=4289c270585445a6aea511da436f75be)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:09.238754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503278b-c7d1-4056-ab27-9f84b06415cc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:09.240370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 591/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:46:09 2022] GET /identity/v3/projects/4289c270585445a6aea511da436f75be => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:09.246491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:09.335094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:09.341174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb9bc9-6e94-40c4-ab24-2042b78f882d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:09.373543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5836ec6-ec9b-4fc9-9dbc-b0277b5102ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:09.374131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5836ec6-ec9b-4fc9-9dbc-b0277b5102ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:09.374407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5836ec6-ec9b-4fc9-9dbc-b0277b5102ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:09.374680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5836ec6-ec9b-4fc9-9dbc-b0277b5102ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:09.375387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5836ec6-ec9b-4fc9-9dbc-b0277b5102ce None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=4289c270585445a6aea511da436f75be, user.id=482d3d075fc34691a5f6408fc77954d7) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:09.375691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5836ec6-ec9b-4fc9-9dbc-b0277b5102ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:09.377097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5836ec6-ec9b-4fc9-9dbc-b0277b5102ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:09.383070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 570/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:46:09 2022] GET /identity/v3/role_assignments?user.id=482d3d075fc34691a5f6408fc77954d7&scope.project.id=4289c270585445a6aea511da436f75be&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:11.540512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74add0-1a55-452e-bb97-e02ca1194bfd None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:11.541045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74add0-1a55-452e-bb97-e02ca1194bfd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:11.541045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74add0-1a55-452e-bb97-e02ca1194bfd None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:11.542424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 592/1162] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:11.547557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bf40d-46b2-4349-bd22-615d5da31714 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:11.547924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bf40d-46b2-4349-bd22-615d5da31714 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:11.548111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bf40d-46b2-4349-bd22-615d5da31714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:11.566382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-181bf40d-46b2-4349-bd22-615d5da31714 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:11.568119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-181bf40d-46b2-4349-bd22-615d5da31714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:11.631496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 571/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:11.898241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74add0-1a55-452e-bb97-e02ca1194bfd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:11.925790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:11.998075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74add0-1a55-452e-bb97-e02ca1194bfd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:12.032211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:12.032810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:12.033092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:12.033401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:12.098840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:12.100284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:12.125528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:12.195645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 593/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:46:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:46:12.201706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181bf40d-46b2-4349-bd22-615d5da31714 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:12.288390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:12.297899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181bf40d-46b2-4349-bd22-615d5da31714 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:12.329870 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273002-9180-4ab6-bcda-8f8d182e096c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:12.330489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273002-9180-4ab6-bcda-8f8d182e096c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:12.330770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273002-9180-4ab6-bcda-8f8d182e096c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:12.331042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273002-9180-4ab6-bcda-8f8d182e096c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:12.338112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273002-9180-4ab6-bcda-8f8d182e096c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:12.339545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273002-9180-4ab6-bcda-8f8d182e096c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-07273002-9180-4ab6-bcda-8f8d182e096c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:46:12.351971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:46:12.357978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:46:12.359029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 572/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:46:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:46:12.360053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:12.458708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf587b-be82-43c9-b2d6-6d33d6455340 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:12.482955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:12.491799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24c379-5531-4160-bf0c-29fb7e275085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:12.492394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24c379-5531-4160-bf0c-29fb7e275085 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:12.492672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24c379-5531-4160-bf0c-29fb7e275085 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:12.492942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24c379-5531-4160-bf0c-29fb7e275085 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:12.493615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24c379-5531-4160-bf0c-29fb7e275085 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:12.493916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24c379-5531-4160-bf0c-29fb7e275085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:12.495275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24c379-5531-4160-bf0c-29fb7e275085 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:12.502132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 594/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 14:46:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:14.584660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33453-901c-483f-8663-3006db2a23b2 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:14.585255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33453-901c-483f-8663-3006db2a23b2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:14.585255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33453-901c-483f-8663-3006db2a23b2 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:14.586550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 573/1167] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:14.591825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844eaf06-fd1a-4a07-9222-313423d937a1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:14.592239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844eaf06-fd1a-4a07-9222-313423d937a1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:14.592374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844eaf06-fd1a-4a07-9222-313423d937a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:14.610782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-844eaf06-fd1a-4a07-9222-313423d937a1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:14.611975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-844eaf06-fd1a-4a07-9222-313423d937a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:14.676553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 595/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:14.947065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33453-901c-483f-8663-3006db2a23b2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:15.045075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33453-901c-483f-8663-3006db2a23b2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:15.078299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6f313-e737-4f8d-ac1f-5787cb4f1323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:15.078894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6f313-e737-4f8d-ac1f-5787cb4f1323 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:15.079172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6f313-e737-4f8d-ac1f-5787cb4f1323 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:15.079441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6f313-e737-4f8d-ac1f-5787cb4f1323 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:15.080094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6f313-e737-4f8d-ac1f-5787cb4f1323 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:15.081533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6f313-e737-4f8d-ac1f-5787cb4f1323 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:15.096937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 574/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Fri Sep 9 14:46:14 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:46:17.239957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d6d73-3595-46d6-93f0-804e74016898 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:17.240499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d6d73-3595-46d6-93f0-804e74016898 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:17.240499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d6d73-3595-46d6-93f0-804e74016898 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:17.241856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 596/1170] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:17.246974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce6806-5cc6-4574-8676-b92fcb6361a5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:17.247400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce6806-5cc6-4574-8676-b92fcb6361a5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:17.247507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce6806-5cc6-4574-8676-b92fcb6361a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:17.265448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-21ce6806-5cc6-4574-8676-b92fcb6361a5 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:17.266548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-21ce6806-5cc6-4574-8676-b92fcb6361a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:17.269214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:17.330028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 575/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:17.594536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d6d73-3595-46d6-93f0-804e74016898 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:17.691111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d6d73-3595-46d6-93f0-804e74016898 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:17.711107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:17.726072 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9950eb-8d07-4e71-b9b9-b3254655b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:17.726687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9950eb-8d07-4e71-b9b9-b3254655b6fb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:17.726965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9950eb-8d07-4e71-b9b9-b3254655b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:17.727237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9950eb-8d07-4e71-b9b9-b3254655b6fb None admin] PATH_INFO: `/v3/domains/42b0821e33e242fda64ef374e09d73a1` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:17.737353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9950eb-8d07-4e71-b9b9-b3254655b6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=42b0821e33e242fda64ef374e09d73a1)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:17.738787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9950eb-8d07-4e71-b9b9-b3254655b6fb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:17.740353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 597/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Fri Sep 9 14:46:17 2022] GET /identity/v3/domains/42b0821e33e242fda64ef374e09d73a1 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:17.746189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce6806-5cc6-4574-8676-b92fcb6361a5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:17.840847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce6806-5cc6-4574-8676-b92fcb6361a5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:17.872363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95646846-7113-4291-920e-59a5ea499c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:17.872902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95646846-7113-4291-920e-59a5ea499c23 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:17.873177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95646846-7113-4291-920e-59a5ea499c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:17.873476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95646846-7113-4291-920e-59a5ea499c23 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:17.874111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95646846-7113-4291-920e-59a5ea499c23 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:17.875511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95646846-7113-4291-920e-59a5ea499c23 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:17.886474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:17.902431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 576/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:46:17 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:46:20.060769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c80cc1-db5f-4b95-af4a-ec08fda5e150 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:20.061431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c80cc1-db5f-4b95-af4a-ec08fda5e150 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:20.061431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c80cc1-db5f-4b95-af4a-ec08fda5e150 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:20.062672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 598/1174] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:20.067929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be2110-020d-4eb1-a846-fcf0729c0b18 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:20.068343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be2110-020d-4eb1-a846-fcf0729c0b18 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:20.068470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be2110-020d-4eb1-a846-fcf0729c0b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:20.086055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-20be2110-020d-4eb1-a846-fcf0729c0b18 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:20.087220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-20be2110-020d-4eb1-a846-fcf0729c0b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:20.156785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 577/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:20.426542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c80cc1-db5f-4b95-af4a-ec08fda5e150 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:20.528265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c80cc1-db5f-4b95-af4a-ec08fda5e150 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:20.560505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d589-0d47-481a-907f-34301f5d28fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:20.561120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d589-0d47-481a-907f-34301f5d28fb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:20.561422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d589-0d47-481a-907f-34301f5d28fb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:20.561690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d589-0d47-481a-907f-34301f5d28fb None admin] PATH_INFO: `/v3/domains/42b0821e33e242fda64ef374e09d73a1` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:20.562777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d589-0d47-481a-907f-34301f5d28fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=42b0821e33e242fda64ef374e09d73a1)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:20.564196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d589-0d47-481a-907f-34301f5d28fb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:20.565766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 599/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Fri Sep 9 14:46:20 2022] GET /identity/v3/domains/42b0821e33e242fda64ef374e09d73a1 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:20.571536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be2110-020d-4eb1-a846-fcf0729c0b18 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:20.666507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be2110-020d-4eb1-a846-fcf0729c0b18 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:20.688761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:20.701128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207926-3e40-454f-8433-1ac6705464d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:20.701747 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207926-3e40-454f-8433-1ac6705464d0 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:20.702021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207926-3e40-454f-8433-1ac6705464d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:20.702287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207926-3e40-454f-8433-1ac6705464d0 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:20.702934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91207926-3e40-454f-8433-1ac6705464d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:20.704341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91207926-3e40-454f-8433-1ac6705464d0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:20.725214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 578/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:46:20 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:46:22.877617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52218606-dd33-4a70-9ffa-10cf9f973a84 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:22.878163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52218606-dd33-4a70-9ffa-10cf9f973a84 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:22.878163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52218606-dd33-4a70-9ffa-10cf9f973a84 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:22.880104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 600/1178] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:22.885228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497ce46-9477-45e5-9c6c-c45f21b73891 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:22.885664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497ce46-9477-45e5-9c6c-c45f21b73891 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:22.885758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497ce46-9477-45e5-9c6c-c45f21b73891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:22.903518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9497ce46-9477-45e5-9c6c-c45f21b73891 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:22.904672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9497ce46-9477-45e5-9c6c-c45f21b73891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:22.967128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 579/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:23.239747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52218606-dd33-4a70-9ffa-10cf9f973a84 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:23.288682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:23.337940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52218606-dd33-4a70-9ffa-10cf9f973a84 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:23.371004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:23.371617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:23.371894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:23.372164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:23.381886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:46:23.386614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:46:23.387964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:46:23.388986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 601/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:46:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:46:23.393807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497ce46-9477-45e5-9c6c-c45f21b73891 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:23.489913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497ce46-9477-45e5-9c6c-c45f21b73891 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:23.498627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:23.523572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:23.524168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:23.524441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:23.524706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:23.525383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:23.525678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:23.527023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:23.534668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 580/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:46:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:23.541043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:23.636493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04be47-39b3-4adc-8805-364d2c2f1ecb None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:23.646168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:23.669804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:23.670407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:23.670681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:23.670950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] PATH_INFO: `/v3/users/c85d2ebbfd8b49e0b891470f28aeb969` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:23.683344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] RBAC: Authorizing `identity:get_user(user_id=c85d2ebbfd8b49e0b891470f28aeb969)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:23.684772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:23.686348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 602/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:46:23 2022] GET /identity/v3/users/c85d2ebbfd8b49e0b891470f28aeb969 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:23.692422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:23.786283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d425f-d642-4385-b2da-1daa64cf1308 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:23.817504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c319a-2772-4d30-8744-94758965e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:23.818096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c319a-2772-4d30-8744-94758965e999 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:23.818368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c319a-2772-4d30-8744-94758965e999 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:23.818633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c319a-2772-4d30-8744-94758965e999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:23.854950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:23.882563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c319a-2772-4d30-8744-94758965e999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:23.883970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c319a-2772-4d30-8744-94758965e999 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:23.975319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 581/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:46:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:46:23.981184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:24.077043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb37bd4-fb89-4c5c-b395-63049c89db59 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:24.109068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3b6f-6a77-464a-b4ee-a6e06bb8c5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:24.109709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3b6f-6a77-464a-b4ee-a6e06bb8c5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:24.109985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3b6f-6a77-464a-b4ee-a6e06bb8c5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:24.110251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3b6f-6a77-464a-b4ee-a6e06bb8c5f7 None admin] PATH_INFO: `/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:24.111332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b3b6f-6a77-464a-b4ee-a6e06bb8c5f7 None admin] RBAC: Authorizing `identity:get_project(project_id=f82ed3ab9a2c46a8aee6f51b7ed1094f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:24.112742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b3b6f-6a77-464a-b4ee-a6e06bb8c5f7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:24.114899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 603/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:46:23 2022] GET /identity/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:24.120845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c319a-2772-4d30-8744-94758965e999 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:24.186098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:24.215884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c319a-2772-4d30-8744-94758965e999 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:24.247372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa769c-b81f-4a49-ab33-e11eb953ac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:24.247958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa769c-b81f-4a49-ab33-e11eb953ac5f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:24.248232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa769c-b81f-4a49-ab33-e11eb953ac5f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:24.248500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa769c-b81f-4a49-ab33-e11eb953ac5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:24.249231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa769c-b81f-4a49-ab33-e11eb953ac5f None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=f82ed3ab9a2c46a8aee6f51b7ed1094f, user.id=c85d2ebbfd8b49e0b891470f28aeb969) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:24.249530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa769c-b81f-4a49-ab33-e11eb953ac5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:24.250950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa769c-b81f-4a49-ab33-e11eb953ac5f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:24.256608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 582/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:46:24 2022] GET /identity/v3/role_assignments?user.id=c85d2ebbfd8b49e0b891470f28aeb969&scope.project.id=f82ed3ab9a2c46a8aee6f51b7ed1094f&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:26.357289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbe94a-1b42-4e46-a0c6-e5ee4f46b13d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:26.357806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbe94a-1b42-4e46-a0c6-e5ee4f46b13d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:26.357806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbe94a-1b42-4e46-a0c6-e5ee4f46b13d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:26.359088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 604/1186] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:26.364140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f5000-db83-43eb-9d86-2e7001ccc7a8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:26.364573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f5000-db83-43eb-9d86-2e7001ccc7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:26.364677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f5000-db83-43eb-9d86-2e7001ccc7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:26.383275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6e1f5000-db83-43eb-9d86-2e7001ccc7a8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:26.384440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6e1f5000-db83-43eb-9d86-2e7001ccc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:26.448318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 583/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:26.716638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbe94a-1b42-4e46-a0c6-e5ee4f46b13d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:26.750303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:26.817389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbe94a-1b42-4e46-a0c6-e5ee4f46b13d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:26.849652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:26.850261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:26.850536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:26.850804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:26.860412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:46:26.865150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:46:26.866235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:46:26.867263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 605/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:46:26.872228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f5000-db83-43eb-9d86-2e7001ccc7a8 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:26.931148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:26.967828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f5000-db83-43eb-9d86-2e7001ccc7a8 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:27.000037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:27.000629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:27.000905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:27.001175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:27.001846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:27.002142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:27.003485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:27.011746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 584/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:27.017808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:27.106024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:27.112853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b36d93-3690-4786-8b02-48a2439bb27d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:27.145844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:27.146446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:27.146722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:27.146990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] PATH_INFO: `/v3/users/c85d2ebbfd8b49e0b891470f28aeb969` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:27.148086 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] RBAC: Authorizing `identity:get_user(user_id=c85d2ebbfd8b49e0b891470f28aeb969)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:27.149660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:27.151188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 606/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:46:27 2022] GET /identity/v3/users/c85d2ebbfd8b49e0b891470f28aeb969 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:27.156858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:27.251929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c5b96-01dc-4215-ad9f-16bb476b3735 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:27.276352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:27.284790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61397b5-bd8d-42d3-bc07-119711b1829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:27.285431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61397b5-bd8d-42d3-bc07-119711b1829c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:27.285712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61397b5-bd8d-42d3-bc07-119711b1829c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:27.285982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61397b5-bd8d-42d3-bc07-119711b1829c None admin] PATH_INFO: `/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:27.287086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61397b5-bd8d-42d3-bc07-119711b1829c None admin] RBAC: Authorizing `identity:get_project(project_id=f82ed3ab9a2c46a8aee6f51b7ed1094f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:27.288499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61397b5-bd8d-42d3-bc07-119711b1829c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:27.290124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 585/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:46:27 2022] GET /identity/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:27.295778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:27.392743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339612b5-1a2a-408e-8366-edc65f018833 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:27.424629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91d44f-87b7-4ac5-8fc5-a028e402dc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:27.425308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d44f-87b7-4ac5-8fc5-a028e402dc41 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:27.425590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d44f-87b7-4ac5-8fc5-a028e402dc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:27.426457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d44f-87b7-4ac5-8fc5-a028e402dc41 None admin] PATH_INFO: `/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f/users/c85d2ebbfd8b49e0b891470f28aeb969/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:27.428345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91d44f-87b7-4ac5-8fc5-a028e402dc41 None admin] RBAC: Authorizing `identity:create_grant(project_id=f82ed3ab9a2c46a8aee6f51b7ed1094f, user_id=c85d2ebbfd8b49e0b891470f28aeb969, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:27.429809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91d44f-87b7-4ac5-8fc5-a028e402dc41 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:27.441849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 607/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:46:27 2022] PUT /identity/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f/users/c85d2ebbfd8b49e0b891470f28aeb969/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:46:29.525564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b69b6d-492e-4e5c-bfaa-9c252d1aa9e7 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:29.526126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b69b6d-492e-4e5c-bfaa-9c252d1aa9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:29.526126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b69b6d-492e-4e5c-bfaa-9c252d1aa9e7 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:29.527426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 586/1193] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:46:29.532817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5f1f1-753e-48bf-9ae4-a3a99b6137ca None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:29.533242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5f1f1-753e-48bf-9ae4-a3a99b6137ca None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:29.533401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5f1f1-753e-48bf-9ae4-a3a99b6137ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:29.551663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-75b5f1f1-753e-48bf-9ae4-a3a99b6137ca None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:46:29.552839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-75b5f1f1-753e-48bf-9ae4-a3a99b6137ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:46:29.568525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:29.617803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 608/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:46:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:46:29.889720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b69b6d-492e-4e5c-bfaa-9c252d1aa9e7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:29.986984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b69b6d-492e-4e5c-bfaa-9c252d1aa9e7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:30.020594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:30.021239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:30.021518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:30.021787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:30.031516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:46:30.036242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:46:30.037449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:46:30.038309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 587/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:46:30.043292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5f1f1-753e-48bf-9ae4-a3a99b6137ca None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:30.137491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5f1f1-753e-48bf-9ae4-a3a99b6137ca None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:30.168967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:30.169523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:30.169798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:30.170066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:30.170709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:30.171011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:30.172360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:30.180438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 609/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:46:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:30.186623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:30.239066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:30.280913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8306d9d-583f-4d77-b03e-c517a5cc02d1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:30.312634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:30.313257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:30.313534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:30.313802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] PATH_INFO: `/v3/users/c85d2ebbfd8b49e0b891470f28aeb969` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:30.314897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] RBAC: Authorizing `identity:get_user(user_id=c85d2ebbfd8b49e0b891470f28aeb969)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:30.316306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:30.318393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 588/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 14:46:30 2022] GET /identity/v3/users/c85d2ebbfd8b49e0b891470f28aeb969 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:30.323908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:30.339333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:30.420813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ec18e-0109-4237-a0a5-ce688a35f7d2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:30.452940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:30.453557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:30.453837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:30.454106 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:30.524647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:30.526143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:30.541290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:30.621221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 610/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:46:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:46:30.627159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:30.720782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729984c-4b8d-4039-9141-8cf80389aabe None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:30.742857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:30.754314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6f08-d24d-489d-b092-d59aaed6286e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:30.754928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6f08-d24d-489d-b092-d59aaed6286e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:30.755203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6f08-d24d-489d-b092-d59aaed6286e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:30.755472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6f08-d24d-489d-b092-d59aaed6286e None admin] PATH_INFO: `/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:30.756566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6f08-d24d-489d-b092-d59aaed6286e None admin] RBAC: Authorizing `identity:get_project(project_id=f82ed3ab9a2c46a8aee6f51b7ed1094f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:30.758030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6f08-d24d-489d-b092-d59aaed6286e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:30.759615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 589/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 14:46:30 2022] GET /identity/v3/projects/f82ed3ab9a2c46a8aee6f51b7ed1094f => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:30.765921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:46:30.865217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c693ebe-1541-4347-be3d-03bad508de70 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:46:30.883872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:30.904524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2094d-2ca1-48f1-a685-e198c87818fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:46:30.905126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2094d-2ca1-48f1-a685-e198c87818fc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:46:30.905439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2094d-2ca1-48f1-a685-e198c87818fc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:46:30.905713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2094d-2ca1-48f1-a685-e198c87818fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:46:30.906426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2094d-2ca1-48f1-a685-e198c87818fc None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=f82ed3ab9a2c46a8aee6f51b7ed1094f, user.id=c85d2ebbfd8b49e0b891470f28aeb969) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:46:30.906731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2094d-2ca1-48f1-a685-e198c87818fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:46:30.908163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2094d-2ca1-48f1-a685-e198c87818fc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:46:30.914302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 611/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:46:30 2022] GET /identity/v3/role_assignments?user.id=c85d2ebbfd8b49e0b891470f28aeb969&scope.project.id=f82ed3ab9a2c46a8aee6f51b7ed1094f&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:46:40.750206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:46:40.911784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=11 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:26.435277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c110b4-ccc5-4350-9e82-452133aed29a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:26.435910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c110b4-ccc5-4350-9e82-452133aed29a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:26.435910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c110b4-ccc5-4350-9e82-452133aed29a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:26.437359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 590/1201] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:26.443250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea39cd5-0c09-4d78-8e4b-965818a8aa5f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:26.443748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea39cd5-0c09-4d78-8e4b-965818a8aa5f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:26.443861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea39cd5-0c09-4d78-8e4b-965818a8aa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:26.465129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3ea39cd5-0c09-4d78-8e4b-965818a8aa5f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:26.466558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3ea39cd5-0c09-4d78-8e4b-965818a8aa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:26.572469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 612/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:47:26.862263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c110b4-ccc5-4350-9e82-452133aed29a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:26.989010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c110b4-ccc5-4350-9e82-452133aed29a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:27.038979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:27.039638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:27.039916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:27.040186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:27.051189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:27.056086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:27.072198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:27.073336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:27.078422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:27.078422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:27.078422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:27.078422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 591/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:47:27.079605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea39cd5-0c09-4d78-8e4b-965818a8aa5f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:27.196682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea39cd5-0c09-4d78-8e4b-965818a8aa5f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:27.233651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:27.240525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8dd705-97c7-4b5d-be6c-8d21510a27d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:27.241878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd705-97c7-4b5d-be6c-8d21510a27d4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:27.242355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd705-97c7-4b5d-be6c-8d21510a27d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:27.242816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd705-97c7-4b5d-be6c-8d21510a27d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:27.243644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd705-97c7-4b5d-be6c-8d21510a27d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:27.244133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd705-97c7-4b5d-be6c-8d21510a27d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:27.245895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd705-97c7-4b5d-be6c-8d21510a27d4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:27.264762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 613/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:27.266001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:27.344100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:27.398774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d0fb-48c9-462c-987d-f786e372c73c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:27.438191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c2fc1-ec70-4c4e-8989-42fed02349bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:27.438884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c2fc1-ec70-4c4e-8989-42fed02349bd None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:27.439073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c2fc1-ec70-4c4e-8989-42fed02349bd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:27.439341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c2fc1-ec70-4c4e-8989-42fed02349bd None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:27.440496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c2fc1-ec70-4c4e-8989-42fed02349bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:27.442104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c2fc1-ec70-4c4e-8989-42fed02349bd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:27.465876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 592/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 14:47:27 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:47:29.821297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed433a4d-744c-4282-9ee4-7910664f614c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:29.821297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed433a4d-744c-4282-9ee4-7910664f614c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:29.821297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed433a4d-744c-4282-9ee4-7910664f614c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:29.821297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 614/1206] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:29.827020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a2bc-feba-431f-becc-ddf580fed717 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:29.827511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a2bc-feba-431f-becc-ddf580fed717 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:29.827767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a2bc-feba-431f-becc-ddf580fed717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:29.849595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bb10a2bc-feba-431f-becc-ddf580fed717 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:29.850891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bb10a2bc-feba-431f-becc-ddf580fed717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:29.939974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 593/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:47:30.225833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed433a4d-744c-4282-9ee4-7910664f614c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:30.347576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed433a4d-744c-4282-9ee4-7910664f614c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:30.385015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:30.385642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:30.385919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:30.386227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:30.397985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:47:30.402948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:30.403925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:47:30.404814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:30.409388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 615/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:47:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:30.410486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10a2bc-feba-431f-becc-ddf580fed717 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:30.456480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:30.535032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10a2bc-feba-431f-becc-ddf580fed717 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:30.573640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:30.574269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:30.574544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:30.574812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:30.575467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:30.575843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:30.577344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:30.587976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 594/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:47:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:30.592986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:30.652743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:30.708550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f75d58-93b2-4caa-ba9e-ba725451cc28 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:30.750942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:30.751576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:30.751867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:30.752838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:30.764986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:30.766663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:30.782358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:30.783757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:30.784668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:30.784668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:30.784668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:30.788757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 616/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:30.789683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:30.896238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:30.899405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fb9f2-1a62-4ddf-a16a-e9600746ec2f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:30.938101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:30.938698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:30.938947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:30.939227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:30.940574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:30.940903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:30.942386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:30.955423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 595/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:47:30.961274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:31.114080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac3d3-b69c-40e8-a50a-b21e0384a07d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:31.115826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:31.153734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:31.154378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:31.154653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:31.154947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:31.163870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:31.165386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:47:31.179496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:31.180508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:31.186281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 617/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:47:31.186896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:31.292690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7327c5-429e-44bf-aee4-707be35e3985 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:31.330267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:31.330941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:31.331265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:31.331626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:31.332590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:31.332901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:31.334634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:31.354809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 596/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 14:47:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:31.360704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:31.475876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c459738-e277-488e-9f27-634394c7d04c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:31.520337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:31.521191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:31.521655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:31.522037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:31.557542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:31.599700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:31.601299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:31.711915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 618/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:47:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 14:47:31.713489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:31.734528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:31.831357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ba583-f55a-469f-93d2-f3673baa589a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:31.869708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:31.870619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:31.871144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:31.871564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:31.883577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:31.885219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:31.901767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:31.908978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:31.910132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:31.910132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:31.910132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:31.910132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 597/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:47:31.910410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:31.983149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:32.028517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150610d-011a-4504-a6c0-03476020f36e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:32.068932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:32.069938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:32.070223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:32.070566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:32.071205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:32.071549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:32.073036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:32.086444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 619/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:32.092239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:32.174991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:32.214100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d304b-85e2-4958-ad3d-93ede2be0433 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:32.259349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66199be6-a185-49da-8809-f81546a541ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:32.260400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66199be6-a185-49da-8809-f81546a541ed None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:32.260874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66199be6-a185-49da-8809-f81546a541ed None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:32.261220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66199be6-a185-49da-8809-f81546a541ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:32.270090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66199be6-a185-49da-8809-f81546a541ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:32.271618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66199be6-a185-49da-8809-f81546a541ed None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-66199be6-a185-49da-8809-f81546a541ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:47:32.284380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:32.285543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:32.286596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 598/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:47:32.293347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:32.425439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a784a-4d2d-4acb-938a-dd2d5c131480 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:32.449376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:32.461926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260c2f9-c099-4031-a98b-9c0cd32c9f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:32.462536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c2f9-c099-4031-a98b-9c0cd32c9f3e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:32.462822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c2f9-c099-4031-a98b-9c0cd32c9f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:32.463088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c2f9-c099-4031-a98b-9c0cd32c9f3e None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:32.463774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260c2f9-c099-4031-a98b-9c0cd32c9f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:32.464075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260c2f9-c099-4031-a98b-9c0cd32c9f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:32.465550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260c2f9-c099-4031-a98b-9c0cd32c9f3e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:32.478244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 620/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:47:32.484988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66199be6-a185-49da-8809-f81546a541ed None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:32.605366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66199be6-a185-49da-8809-f81546a541ed None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:32.638679 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:32.645851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566a5de-f5b0-4c3b-8ab4-77109b661b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:32.646585 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566a5de-f5b0-4c3b-8ab4-77109b661b16 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:32.646963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566a5de-f5b0-4c3b-8ab4-77109b661b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:32.647235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566a5de-f5b0-4c3b-8ab4-77109b661b16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:32.648119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566a5de-f5b0-4c3b-8ab4-77109b661b16 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=57cca115ce044ff5aacd5ea1a8834402) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:32.648441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566a5de-f5b0-4c3b-8ab4-77109b661b16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:32.650184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566a5de-f5b0-4c3b-8ab4-77109b661b16 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:32.656441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 599/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:47:32 2022] GET /identity/v3/role_assignments?user.id=57cca115ce044ff5aacd5ea1a8834402&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:34.830949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278794-ebfb-4a05-93c8-b30556e3e0f4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:34.831502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278794-ebfb-4a05-93c8-b30556e3e0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:34.831502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278794-ebfb-4a05-93c8-b30556e3e0f4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:34.832938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 621/1220] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:34.838496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5122c2-4a59-41be-b807-2d9a91ef66b8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:34.839010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5122c2-4a59-41be-b807-2d9a91ef66b8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:34.839010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5122c2-4a59-41be-b807-2d9a91ef66b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:34.857668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3c5122c2-4a59-41be-b807-2d9a91ef66b8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:34.858866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3c5122c2-4a59-41be-b807-2d9a91ef66b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:34.932251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 600/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:47:35.254102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278794-ebfb-4a05-93c8-b30556e3e0f4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:35.373218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278794-ebfb-4a05-93c8-b30556e3e0f4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:35.413800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:35.414476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:35.414968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:35.415330 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:35.429360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:47:35.434419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:35.435583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:47:35.436552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:35.437807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 622/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:47:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:35.443231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5122c2-4a59-41be-b807-2d9a91ef66b8 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:35.562651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5122c2-4a59-41be-b807-2d9a91ef66b8 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:35.589644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:35.601655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:35.602262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:35.602537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:35.602803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:35.603454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:35.603753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:35.605189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:35.614591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 601/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:47:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:35.621449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:35.670103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:35.727184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb97bc4-af11-4ec2-ad25-807d770494ac None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:35.764826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:35.765476 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:35.765750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:35.766016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:35.778425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:35.780129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:35.797954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:35.799647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:35.800609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:35.800609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:35.800609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:35.800609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 623/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:35.806867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:35.921175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b9544-2b2a-4787-9eff-21a362a4c99b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:35.960807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:35.961505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:35.961816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:35.962122 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:35.962809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:35.963093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:35.964601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:35.977793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 602/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:35.984034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:36.089359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:36.097517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6ff38-298e-41ca-b264-4eae56a0db2d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:36.134304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:36.134941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:36.135248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:36.135592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:36.144389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:36.145928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:47:36.157978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:36.159701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:36.161229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 624/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:36.165895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:36.215544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:36.276822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb04e7-9cd1-47e0-9e4c-50e06ddb3213 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:36.312963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:36.313598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:36.313881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:36.314137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:36.314874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:36.315172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:36.316636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:36.337840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 603/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 14:47:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:36.343647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:36.454085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6052472-ecf4-4fed-8ce2-b345f54625c4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:36.495690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:36.496253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:36.496610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:36.496944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:36.511658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:36.513756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:36.530258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:36.531529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:36.532481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:36.532481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:36.532481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:36.532481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 625/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:36.538134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:36.647646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61dc5d-b39a-4c06-8e30-61b3875c0bc1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:36.650346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:36.685281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:36.685885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:36.686162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:36.686432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:36.687065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:36.687368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:36.688733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:36.701430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 604/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:36.710551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:36.719858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:36.823446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d817c-ef1a-4143-8b87-69ad66315609 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:36.860907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:36.862144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:36.862444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:36.862710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:36.869875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:36.871480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:47:36.884294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:36.885625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:36.886788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 626/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:36.891633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:37.004386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965855ae-38c8-4a17-83f9-23a8aefc0950 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:37.046007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2038f-c9e4-4d43-8d91-46dbb1b58db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:37.046616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2038f-c9e4-4d43-8d91-46dbb1b58db5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:37.047133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2038f-c9e4-4d43-8d91-46dbb1b58db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:37.047418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2038f-c9e4-4d43-8d91-46dbb1b58db5 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:37.048237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2038f-c9e4-4d43-8d91-46dbb1b58db5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:37.048537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2038f-c9e4-4d43-8d91-46dbb1b58db5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:37.050967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2038f-c9e4-4d43-8d91-46dbb1b58db5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:37.064250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 605/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:37.071843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:37.139449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:37.178785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02f774-f4ab-4d42-b763-c3966fd4a992 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:37.218275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aaeb8-828b-4181-9bf6-fae63409c5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:37.218937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aaeb8-828b-4181-9bf6-fae63409c5d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:37.219220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aaeb8-828b-4181-9bf6-fae63409c5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:37.219498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aaeb8-828b-4181-9bf6-fae63409c5d5 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/57cca115ce044ff5aacd5ea1a8834402/roles/3a9af0f0d1d04401a0fee6860911b19f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:37.235888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2aaeb8-828b-4181-9bf6-fae63409c5d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=57cca115ce044ff5aacd5ea1a8834402, role_id=3a9af0f0d1d04401a0fee6860911b19f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:37.238207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2aaeb8-828b-4181-9bf6-fae63409c5d5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:37.253957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 627/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:47:37 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/57cca115ce044ff5aacd5ea1a8834402/roles/3a9af0f0d1d04401a0fee6860911b19f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:47:39.567381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6eebe6-dd96-40cc-992a-8d98b2291a22 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:39.567941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6eebe6-dd96-40cc-992a-8d98b2291a22 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:39.567941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6eebe6-dd96-40cc-992a-8d98b2291a22 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:39.569287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 606/1233] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:39.574486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae5a40-4bba-48cd-8c10-990acbf2e9fa None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:39.574873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae5a40-4bba-48cd-8c10-990acbf2e9fa None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:39.575047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae5a40-4bba-48cd-8c10-990acbf2e9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:39.595170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e1ae5a40-4bba-48cd-8c10-990acbf2e9fa None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:39.596815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e1ae5a40-4bba-48cd-8c10-990acbf2e9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:39.684544 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 628/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:47:39.976875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6eebe6-dd96-40cc-992a-8d98b2291a22 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:40.007226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:40.095975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6eebe6-dd96-40cc-992a-8d98b2291a22 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:40.139021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:40.139663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:40.139967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:40.140245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:40.150516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:47:40.155330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:40.156360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:40.157610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 607/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 14:47:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:40.163463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae5a40-4bba-48cd-8c10-990acbf2e9fa None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:40.211572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:40.265955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae5a40-4bba-48cd-8c10-990acbf2e9fa None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:40.303940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:40.304548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:40.304825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:40.305094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:40.305779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:40.306077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:40.307674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:40.319638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 629/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 14:47:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:40.326252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:40.429854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:40.448967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aba30-762b-4149-9df5-0e7a9bf7d158 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:40.487193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:40.488052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:40.488177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:40.488447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:40.499464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:40.501008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:40.518246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:40.520310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:40.522019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:40.522019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:40.522019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:40.522019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 608/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:47:40.526941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:40.643971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0a70-98bd-4fe4-83a8-e401ef32b30f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:40.653784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:40.683204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:40.683816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:40.684095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:40.684365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:40.684995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:40.685321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:40.686742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:40.699274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 630/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:40.706465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:40.820374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8526e-115b-4a9f-9d4c-48988cfe3871 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:40.859491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:40.860120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:40.860400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:40.860673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:40.863851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:40.870335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:40.871947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:47:40.883974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:40.884938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:40.886013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 609/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:40.891272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:40.999226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e80aa-834f-4b07-a667-57be9d31c575 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:41.037967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:41.038634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:41.038915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:41.039221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:41.039903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:41.040203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:41.041752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:41.059029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 631/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 14:47:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:41.065382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:41.175106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b87d6b-ffc8-46d6-a916-733520facb9d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:41.212832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:41.213552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:41.213852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:41.214152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:41.283760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:41.291334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:41.292839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:41.400484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 610/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:47:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:47:41.406479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:41.435224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:41.521971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459fe00-5a23-4753-aa34-bf840073fb19 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:41.566937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:41.567616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:41.567898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:41.568175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:41.579289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:41.580783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:41.596857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:41.598074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:41.599199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:41.599199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:41.599199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:41.599498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 632/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 14:47:41.605303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:41.715424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:41.722720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c4dd0-3672-40c0-908f-2fa4c9ae051a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:41.763778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:41.764638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:41.764925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:41.765295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:41.766199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:41.766620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:41.768245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:41.780141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 611/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:41.786430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:41.874711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:41.898689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3bb3-6a6a-446f-9d34-f3fba8b3a94d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:41.936113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:41.936739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:41.937015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:41.937310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:41.947461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:41.949312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:47:41.962943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:41.964050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:41.969907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 633/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:47:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:41.971100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:42.094809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165547e-2a50-4f62-82e2-70f6c2c1aa72 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:42.131306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197686-ec4b-470a-bea4-7b651e783eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:42.131901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197686-ec4b-470a-bea4-7b651e783eab None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:42.132173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197686-ec4b-470a-bea4-7b651e783eab None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:42.132470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197686-ec4b-470a-bea4-7b651e783eab None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:42.133117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197686-ec4b-470a-bea4-7b651e783eab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:42.133443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197686-ec4b-470a-bea4-7b651e783eab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:42.134845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197686-ec4b-470a-bea4-7b651e783eab None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:42.147388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 612/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:47:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:47:42.153823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:42.265087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f746-9fed-41f9-bad0-98cb92149602 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:42.302531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f93233-49f1-4788-89e8-fdcbaf3ea080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:42.303145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93233-49f1-4788-89e8-fdcbaf3ea080 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:42.303434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93233-49f1-4788-89e8-fdcbaf3ea080 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:42.303701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93233-49f1-4788-89e8-fdcbaf3ea080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:42.304426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f93233-49f1-4788-89e8-fdcbaf3ea080 None admin] RBAC: Adding query filter params (role.id=3a9af0f0d1d04401a0fee6860911b19f, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=57cca115ce044ff5aacd5ea1a8834402) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:42.304724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f93233-49f1-4788-89e8-fdcbaf3ea080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:42.306195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f93233-49f1-4788-89e8-fdcbaf3ea080 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:42.321450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 634/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:47:42 2022] GET /identity/v3/role_assignments?user.id=57cca115ce044ff5aacd5ea1a8834402&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=3a9af0f0d1d04401a0fee6860911b19f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:44.590504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380c1c-1aa4-4945-8d16-4179442c45dd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:44.591084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380c1c-1aa4-4945-8d16-4179442c45dd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:44.591084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380c1c-1aa4-4945-8d16-4179442c45dd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:44.592378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 613/1247] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:44.597854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc024f-9fc8-4585-9635-97bd6ea9733e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:44.598283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc024f-9fc8-4585-9635-97bd6ea9733e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:44.598425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc024f-9fc8-4585-9635-97bd6ea9733e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:44.604655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:44.617844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-04cc024f-9fc8-4585-9635-97bd6ea9733e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:44.619136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-04cc024f-9fc8-4585-9635-97bd6ea9733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:44.694381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 635/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:47:45.003683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0380c1c-1aa4-4945-8d16-4179442c45dd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:45.024593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:45.118039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0380c1c-1aa4-4945-8d16-4179442c45dd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:45.155389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:45.156258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:45.156447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:45.156748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:45.168580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:47:45.173674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:45.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:45.175707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 614/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:47:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:45.180929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc024f-9fc8-4585-9635-97bd6ea9733e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:45.301822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc024f-9fc8-4585-9635-97bd6ea9733e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:45.336225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:45.355776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:45.356390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:45.356665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:45.356931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:45.357649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:45.357971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:45.359457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:45.368365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 636/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:47:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:45.374343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:45.460955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:45.487151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab64e-52f5-4fe2-865e-66ce1faf7996 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:45.538879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:45.539823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:45.540140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:45.540574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:45.555547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:45.556939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:45.574692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:45.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:45.576770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:45.576770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:45.576770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:45.576770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 615/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:45.582820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:45.694386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e96f52-b0fc-46c9-bec8-02da2932138a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:45.730142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:45.730769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:45.731050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:45.731325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:45.731981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:45.732288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:45.733748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:45.746913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 637/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:45.753539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:45.856983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3e2b-5ebc-4788-a3ed-663b56f9fcab None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:45.884227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:45.893568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:45.894210 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:45.894489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:45.894762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:45.902717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:45.904159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:47:45.915615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:45.916789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:45.917966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 616/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:45.925222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:45.983477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:46.036292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888241d-9c5f-4e5f-8dcc-335b4bf5fc01 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:46.071948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:46.072560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:46.072841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:46.073113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:46.073870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:46.074173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:46.075612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:46.092547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 638/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 14:47:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:46.098542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:46.196428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29dce-e530-470d-9185-c44165a76392 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:46.229001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:46.229627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:46.229905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:46.230174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:46.282641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:46.295296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:46.296739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:46.393453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 617/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:47:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:47:46.399184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:46.498240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a69b2-9888-4885-9392-5aeb772c9ba1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:46.513656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:46.538314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:46.538938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:46.539213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:46.539481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:46.549697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:46.551132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:46.565773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:46.566874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:46.568158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:46.568158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:46.568158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:46.568158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 639/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:46.572571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:46.655552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:46.667153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843bf94-6bf9-404b-9dc5-4ac016beeab8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:46.699541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:46.700125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:46.700398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:46.700666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:46.701324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:46.701622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:46.702970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:46.713586 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 618/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:46.719110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:46.814141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02867aa-0edc-412a-830f-eb2b40ef9c57 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:46.846681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:46.847295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:46.847569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:46.847835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:46.854166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:46.855596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:47:46.867083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:46.868134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:46.869252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 640/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:46.873930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:46.968960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f6feb-e483-4f81-8593-e81bd4e505c6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:47.001096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c808d1-46a8-483a-8e65-c35357fe9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:47.001711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c808d1-46a8-483a-8e65-c35357fe9d34 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:47.001987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c808d1-46a8-483a-8e65-c35357fe9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:47.002737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c808d1-46a8-483a-8e65-c35357fe9d34 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:47.003446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c808d1-46a8-483a-8e65-c35357fe9d34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:47.003758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c808d1-46a8-483a-8e65-c35357fe9d34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:47.005171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c808d1-46a8-483a-8e65-c35357fe9d34 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:47.017181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 619/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:47:47.023722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:47.048513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:47.120114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce04624-a7d1-44d0-be87-35bbd4e4458d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:47.152604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac1a3f-e7f6-4bbc-a3f6-29b0a088db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:47.153189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac1a3f-e7f6-4bbc-a3f6-29b0a088db84 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:47.153494 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac1a3f-e7f6-4bbc-a3f6-29b0a088db84 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:47.153760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac1a3f-e7f6-4bbc-a3f6-29b0a088db84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:47.154465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac1a3f-e7f6-4bbc-a3f6-29b0a088db84 None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=57cca115ce044ff5aacd5ea1a8834402) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:47.154767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac1a3f-e7f6-4bbc-a3f6-29b0a088db84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:47.156190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac1a3f-e7f6-4bbc-a3f6-29b0a088db84 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:47.161376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 641/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:47:47 2022] GET /identity/v3/role_assignments?user.id=57cca115ce044ff5aacd5ea1a8834402&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=82245735988747a4bc5704727a1cf374 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:49.265281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb37d7a-23d8-4f0d-9b85-e87397f7e224 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:49.265796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb37d7a-23d8-4f0d-9b85-e87397f7e224 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:49.265796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb37d7a-23d8-4f0d-9b85-e87397f7e224 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:49.267221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 620/1261] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:49.272171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bda5b-3b0d-46ab-bfcb-6827ce7f788c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:49.272578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bda5b-3b0d-46ab-bfcb-6827ce7f788c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:49.272726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bda5b-3b0d-46ab-bfcb-6827ce7f788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:49.291025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f77bda5b-3b0d-46ab-bfcb-6827ce7f788c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:49.292209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f77bda5b-3b0d-46ab-bfcb-6827ce7f788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:49.356093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 642/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:47:49.631324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb37d7a-23d8-4f0d-9b85-e87397f7e224 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:49.642409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:49.734153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb37d7a-23d8-4f0d-9b85-e87397f7e224 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:49.766235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:49.766850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:49.767131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:49.767402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:49.777636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:47:49.782411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:49.784106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:49.785223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 621/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:49.790010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bda5b-3b0d-46ab-bfcb-6827ce7f788c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:49.807435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:49.884554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bda5b-3b0d-46ab-bfcb-6827ce7f788c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:49.917034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:49.917650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:49.917927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:49.918196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:49.918849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:49.919146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:49.920502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:49.928130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 643/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:49.933758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:49.998107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:50.031455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7773-4005-4be1-8eed-484dcc7b354b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:50.063697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:50.064316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:50.065082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:50.065414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:50.075051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:50.076469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:50.091006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:50.092112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:50.093505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:50.093505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:50.093505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:50.093505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 622/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:50.097903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:50.177040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:50.195813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ed70a-9128-4c3f-b7d5-c8478540f0a1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:50.228215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:50.228810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:50.229091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:50.229389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:50.230016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:50.230317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:50.231673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:50.243320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 644/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:50.248879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:50.345068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8370ad0-df7e-4966-a3cd-a260cd9ab054 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:50.364227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:50.378514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:50.379129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:50.379408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:50.379678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:50.386825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:50.388499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:47:50.399313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:50.404749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:50.406049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 623/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:50.406782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:50.504752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90efe75-0cf4-41f6-adac-d5cd87a27847 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:50.538229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:50.538819 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:50.539095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:50.539367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:50.540076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:50.540375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:50.541836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:50.550605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:50.558888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 645/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 14:47:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:50.564771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:50.660494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c6533-bcf0-44c6-a9f1-171b12dc2b10 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:50.692192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:50.692794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:50.693067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:50.693365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:50.703245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:50.704650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:50.718079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:50.719158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:50.720036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:50.720036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:50.720036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:50.720036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 624/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:50.725317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:50.822944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1cbf9-bc9e-4024-8c3b-e330e388a3f3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:50.855421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:50.856018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:50.856297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:50.856567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:50.857229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:50.857526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:50.858889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:50.870402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 646/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:50.875991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:50.906243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:50.971895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ade75-9e8c-4f77-ab58-ced7de82cd75 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:51.003445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:51.004053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:51.004325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:51.004589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:51.010955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:51.012383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:47:51.024393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:51.025469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:51.026619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 625/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:51.031157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:51.089510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:51.129772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5075779-0a99-49e8-a166-2b290014db78 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:51.162688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47e0eb-4b3e-434b-9e02-452caf60a904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:51.163279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47e0eb-4b3e-434b-9e02-452caf60a904 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:51.163556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47e0eb-4b3e-434b-9e02-452caf60a904 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:51.163832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47e0eb-4b3e-434b-9e02-452caf60a904 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:51.164508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47e0eb-4b3e-434b-9e02-452caf60a904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:51.164811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47e0eb-4b3e-434b-9e02-452caf60a904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:51.166262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47e0eb-4b3e-434b-9e02-452caf60a904 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:51.177739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 647/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:47:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:51.183657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:51.271091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:51.279600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcf329-d866-4304-aa65-93f5b8403e6b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:51.311944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a31d5-7c59-4a97-be65-74be93ff1135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:51.312578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a31d5-7c59-4a97-be65-74be93ff1135 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:51.312855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a31d5-7c59-4a97-be65-74be93ff1135 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:51.313125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a31d5-7c59-4a97-be65-74be93ff1135 None admin] PATH_INFO: `/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/57cca115ce044ff5aacd5ea1a8834402/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:51.315007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a31d5-7c59-4a97-be65-74be93ff1135 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb92812b7a74c53b321dd9e80955569, user_id=57cca115ce044ff5aacd5ea1a8834402, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:51.316420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a31d5-7c59-4a97-be65-74be93ff1135 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:51.328591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 626/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 14:47:51 2022] PUT /identity/v3/projects/9eb92812b7a74c53b321dd9e80955569/users/57cca115ce044ff5aacd5ea1a8834402/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:47:53.421768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b1fe-78d2-4861-b8f8-e92968b206d4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:53.422326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b1fe-78d2-4861-b8f8-e92968b206d4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:53.422326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b1fe-78d2-4861-b8f8-e92968b206d4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:53.423634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 648/1274] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:53.428723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4d01-fd9b-4c88-b9a5-93ee4b2d7160 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:53.429140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4d01-fd9b-4c88-b9a5-93ee4b2d7160 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:53.429278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4d01-fd9b-4c88-b9a5-93ee4b2d7160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:53.447023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-249e4d01-fd9b-4c88-b9a5-93ee4b2d7160 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:53.448200 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-249e4d01-fd9b-4c88-b9a5-93ee4b2d7160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:53.518600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 627/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:47:53.790515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8b1fe-78d2-4861-b8f8-e92968b206d4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:53.890515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8b1fe-78d2-4861-b8f8-e92968b206d4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:53.908333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:53.924764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:53.925402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:53.925680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:53.925950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:53.935661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 14:47:53.940886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:53.942031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:53.943085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 649/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 14:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:53.949983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e4d01-fd9b-4c88-b9a5-93ee4b2d7160 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:54.029159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:54.048243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e4d01-fd9b-4c88-b9a5-93ee4b2d7160 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:54.080843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:54.081455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:54.081730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:54.081996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:54.082640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:54.082937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:54.084266 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:54.091812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 628/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 14:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:54.097728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:54.194659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf15d6e-395e-4129-8c4d-931cbbaf9434 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:54.226144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:54.226749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:54.227024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:54.227290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:54.237283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:54.238709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:54.253011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:54.254192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:54.255141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:54.255141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:54.255141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:54.255141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 650/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:54.259826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:54.354730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e320f9-a803-469a-95c1-7a936aa01eea None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:54.386526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:54.387111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:54.387387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:54.387922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:54.388549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:54.388853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:54.390226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:54.401343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 629/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:54.406829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:54.436856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:54.504526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5e17a-0352-42fe-a4c8-7dc41845dddc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:54.536711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:54.537347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:54.537624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:54.537891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:54.544898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:54.546359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 14:47:54.557581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:54.558705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:54.559857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 651/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:54.564377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:54.568145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:54.658834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2323505-d4c9-41ea-8558-44c9a4ee65b4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:54.690939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:54.691529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:54.691807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:54.692079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:54.692787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:54.693083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:54.694520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:54.710618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 630/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 14:47:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:54.716118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:54.805208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:54.814835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b4876-aadf-49f2-980a-ff0db332886e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:54.847255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:54.847844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:54.848118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:54.848385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:54.912822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:54.914212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:54.997509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:55.007713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 652/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 14:47:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 14:47:55.014309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:55.074286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:55.111771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940e369-7a0c-485c-aee9-871ac39781d0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:55.144845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:55.145545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:55.145820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:55.146086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:55.155584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:55.157008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 14:47:55.170987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:55.172103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 14:47:55.173035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 14:47:55.173035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 14:47:55.173035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:55.173035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 631/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:55.177809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:55.272861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7e387-8d6c-46f8-9095-9917fc559386 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:55.304635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:55.305236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:55.305514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:55.305783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:55.306406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:55.306702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:55.308048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:55.319214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 653/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 14:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:55.324677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:55.434876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759c5bd-2bc3-4a08-95e3-9095aa1463da None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:55.448508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:55.468461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:55.469083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:55.469389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:55.469663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:55.476079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:55.477536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 14:47:55.489754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:55.490831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Sep 09 14:47:55.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 632/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 14:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:55.496731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:55.544601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:55.594037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7f545-f2dd-4b92-8fe5-299b90575a35 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:55.626283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722735a-2318-440a-abcc-77424fdd7019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:55.626874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722735a-2318-440a-abcc-77424fdd7019 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:55.627739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722735a-2318-440a-abcc-77424fdd7019 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:55.628012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722735a-2318-440a-abcc-77424fdd7019 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:55.628687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722735a-2318-440a-abcc-77424fdd7019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:55.628990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722735a-2318-440a-abcc-77424fdd7019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:55.630444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722735a-2318-440a-abcc-77424fdd7019 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:55.642175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 654/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 14:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:55.648209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:55.750133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332358e-a5bd-48b5-afd7-9a4ee8d2dbd5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:55.783609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d14f2-f901-492a-aa1a-0049b59e45e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:55.784202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d14f2-f901-492a-aa1a-0049b59e45e7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:55.784477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d14f2-f901-492a-aa1a-0049b59e45e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:55.784746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d14f2-f901-492a-aa1a-0049b59e45e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:55.785478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d14f2-f901-492a-aa1a-0049b59e45e7 None admin] RBAC: Adding query filter params (role.id=82245735988747a4bc5704727a1cf374, scope.project.id=9eb92812b7a74c53b321dd9e80955569, user.id=57cca115ce044ff5aacd5ea1a8834402) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:55.785780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d14f2-f901-492a-aa1a-0049b59e45e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:55.787176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d14f2-f901-492a-aa1a-0049b59e45e7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:55.792666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 633/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 14:47:55 2022] GET /identity/v3/role_assignments?user.id=57cca115ce044ff5aacd5ea1a8834402&scope.project.id=9eb92812b7a74c53b321dd9e80955569&role.id=82245735988747a4bc5704727a1cf374 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:57.938866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77ed7c-46f5-4456-b29a-7b38647d70c4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:57.939429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77ed7c-46f5-4456-b29a-7b38647d70c4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:57.939429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77ed7c-46f5-4456-b29a-7b38647d70c4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:57.940788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 655/1288] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:47:57.945996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800b677-e776-4f4d-807d-3761cf7110ec None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:57.946367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800b677-e776-4f4d-807d-3761cf7110ec None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:57.946534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800b677-e776-4f4d-807d-3761cf7110ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:57.964802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0800b677-e776-4f4d-807d-3761cf7110ec None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:47:57.966023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0800b677-e776-4f4d-807d-3761cf7110ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:47:57.994571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:58.031350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 634/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:47:58.295231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77ed7c-46f5-4456-b29a-7b38647d70c4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:58.394343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77ed7c-46f5-4456-b29a-7b38647d70c4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:58.404586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:47:58.427359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:58.427974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:58.428253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:58.428522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:58.429093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:58.430479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:47:58.439014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:47:58.440764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:47:58.441980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 656/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:47:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:47:58.446573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800b677-e776-4f4d-807d-3761cf7110ec None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:58.552901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800b677-e776-4f4d-807d-3761cf7110ec None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:58.585891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74382c4-4759-4bd0-b9d9-e4490419442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:58.586490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74382c4-4759-4bd0-b9d9-e4490419442b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:58.586770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74382c4-4759-4bd0-b9d9-e4490419442b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:58.587040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74382c4-4759-4bd0-b9d9-e4490419442b None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:58.587659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74382c4-4759-4bd0-b9d9-e4490419442b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:58.587958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74382c4-4759-4bd0-b9d9-e4490419442b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:58.589346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74382c4-4759-4bd0-b9d9-e4490419442b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:58.593977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 635/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 14:47:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:47:58.599828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:47:58.696636 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb44bd8-6676-4e3f-aa33-8ef5a7329a9c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:47:58.728037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8032c-6e8c-4a48-9df6-03ab5569400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:47:58.728628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8032c-6e8c-4a48-9df6-03ab5569400f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:47:58.728905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8032c-6e8c-4a48-9df6-03ab5569400f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:47:58.729176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8032c-6e8c-4a48-9df6-03ab5569400f None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:47:58.729826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8032c-6e8c-4a48-9df6-03ab5569400f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:47:58.730125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8032c-6e8c-4a48-9df6-03ab5569400f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:47:58.731483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8032c-6e8c-4a48-9df6-03ab5569400f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:47:58.735739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 657/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 14:47:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:48:00.823677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2503c7-6302-455e-87c8-91b54ec5eee5 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:00.824278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2503c7-6302-455e-87c8-91b54ec5eee5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:00.824278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2503c7-6302-455e-87c8-91b54ec5eee5 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:00.825590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 636/1293] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:48:00.831090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1fcd5-a9c4-4f0f-a26a-ffe80dc38c70 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:00.831467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1fcd5-a9c4-4f0f-a26a-ffe80dc38c70 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:00.831630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1fcd5-a9c4-4f0f-a26a-ffe80dc38c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:00.850008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-acd1fcd5-a9c4-4f0f-a26a-ffe80dc38c70 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:48:00.851193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-acd1fcd5-a9c4-4f0f-a26a-ffe80dc38c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:48:00.879288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:00.916027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 658/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:48:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:48:01.179541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2503c7-6302-455e-87c8-91b54ec5eee5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:01.211409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:01.278941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2503c7-6302-455e-87c8-91b54ec5eee5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:01.311576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e70a7-d08c-4695-aec1-5a4228238d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:01.312165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e70a7-d08c-4695-aec1-5a4228238d67 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:01.312443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e70a7-d08c-4695-aec1-5a4228238d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:01.312717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e70a7-d08c-4695-aec1-5a4228238d67 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:01.313389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e70a7-d08c-4695-aec1-5a4228238d67 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:01.314771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e70a7-d08c-4695-aec1-5a4228238d67 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:01.328176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 637/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 14:48:01 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:48:03.476087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f8cd3-d87e-4508-9086-e43216ea3bc4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:03.476620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f8cd3-d87e-4508-9086-e43216ea3bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:03.476620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f8cd3-d87e-4508-9086-e43216ea3bc4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:03.478016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 659/1296] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:48:03.483598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165b8e-523b-45a1-a1fc-75f221eb72d7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:03.484075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165b8e-523b-45a1-a1fc-75f221eb72d7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:03.484181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165b8e-523b-45a1-a1fc-75f221eb72d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:03.504681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a7165b8e-523b-45a1-a1fc-75f221eb72d7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:48:03.506034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a7165b8e-523b-45a1-a1fc-75f221eb72d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:48:03.583226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 638/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:48:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 14:48:03.846420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f8cd3-d87e-4508-9086-e43216ea3bc4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:03.945486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f8cd3-d87e-4508-9086-e43216ea3bc4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:03.977458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:03.978071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:03.978350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:03.978617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:03.979188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:03.980548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:48:03.989615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:48:03.990760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:48:03.991994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 660/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:48:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:48:03.996562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7165b8e-523b-45a1-a1fc-75f221eb72d7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:04.015654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:04.094118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7165b8e-523b-45a1-a1fc-75f221eb72d7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:04.126624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:04.127901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:04.128178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:04.128443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:04.129052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:48:04.129378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:04.130753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:04.135648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 639/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 14:48:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:48:04.141959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:04.172485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:04.238338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05d67a-c691-4b0e-94b9-508ff9c4fea5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:04.270975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597f0c8-2f2d-4f94-8adb-020dbc8e51cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:04.271568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597f0c8-2f2d-4f94-8adb-020dbc8e51cd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:04.271842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597f0c8-2f2d-4f94-8adb-020dbc8e51cd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:04.272107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597f0c8-2f2d-4f94-8adb-020dbc8e51cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:04.272794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597f0c8-2f2d-4f94-8adb-020dbc8e51cd None admin] RBAC: Adding query filter params (interface=public, service_id=6381b749c2394eababfe70327166011e, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:48:04.273097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597f0c8-2f2d-4f94-8adb-020dbc8e51cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:04.274479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597f0c8-2f2d-4f94-8adb-020dbc8e51cd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:04.279488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 661/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 14:48:04 2022] GET /identity/v3/endpoints?service_id=6381b749c2394eababfe70327166011e&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:48:04.285172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:04.368208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:04.386405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785019-d438-4e4b-b9d4-d65ac5868f41 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:04.419017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bff0a8-1652-4366-9132-577cd2266015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:04.419619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bff0a8-1652-4366-9132-577cd2266015 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:04.419897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bff0a8-1652-4366-9132-577cd2266015 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:04.420165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bff0a8-1652-4366-9132-577cd2266015 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:04.420758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bff0a8-1652-4366-9132-577cd2266015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:04.422131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bff0a8-1652-4366-9132-577cd2266015 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:04.428707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 640/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 14:48:04 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:48:06.541519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451c01c-3971-4a94-ba00-fc9dbd39f84a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:06.542050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451c01c-3971-4a94-ba00-fc9dbd39f84a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:06.542050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451c01c-3971-4a94-ba00-fc9dbd39f84a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:06.543388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 662/1302] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:48:06.549431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059a3a-69cc-4b4b-8daf-f28047799106 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:06.549853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059a3a-69cc-4b4b-8daf-f28047799106 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:06.549977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059a3a-69cc-4b4b-8daf-f28047799106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:06.567676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8d059a3a-69cc-4b4b-8daf-f28047799106 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:48:06.568855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8d059a3a-69cc-4b4b-8daf-f28047799106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:48:06.632214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 641/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:48:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 14:48:06.899686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451c01c-3971-4a94-ba00-fc9dbd39f84a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:06.990940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:06.998206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451c01c-3971-4a94-ba00-fc9dbd39f84a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:07.033631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:07.034255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:07.034541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:07.034803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:07.035380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:07.036738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 14:48:07.045340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 14:48:07.046487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 09 14:48:07.047581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 663/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 14:48:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 14:48:07.052268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059a3a-69cc-4b4b-8daf-f28047799106 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:07.124822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:07.148115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059a3a-69cc-4b4b-8daf-f28047799106 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:07.179848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3bbf3-3f52-4214-8788-f194cfae5202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:07.180384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3bbf3-3f52-4214-8788-f194cfae5202 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:07.180661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3bbf3-3f52-4214-8788-f194cfae5202 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:07.180929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3bbf3-3f52-4214-8788-f194cfae5202 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:07.181568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3bbf3-3f52-4214-8788-f194cfae5202 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:48:07.181866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3bbf3-3f52-4214-8788-f194cfae5202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:07.183209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3bbf3-3f52-4214-8788-f194cfae5202 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:07.188109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 642/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 14:48:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:48:07.194071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:48:07.289207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb082f-c615-459e-961e-cfa429614c6e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:48:07.321434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cff3f-d03e-44c1-857c-d79befe5e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:48:07.322020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cff3f-d03e-44c1-857c-d79befe5e123 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:48:07.322298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cff3f-d03e-44c1-857c-d79befe5e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:48:07.322567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cff3f-d03e-44c1-857c-d79befe5e123 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:48:07.323231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794cff3f-d03e-44c1-857c-d79befe5e123 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:48:07.324593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794cff3f-d03e-44c1-857c-d79befe5e123 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:48:07.343885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 664/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:48:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:48:17.185640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=30 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:48:17.341103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:11.079476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cdb6-3f03-4a0a-8d79-e5078523b245 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:11.079922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cdb6-3f03-4a0a-8d79-e5078523b245 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:11.080371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cdb6-3f03-4a0a-8d79-e5078523b245 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:11.085811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 643/1307] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:11.088121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473956a9-6e1e-4b1e-be4a-78da68ad12c4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:11.088526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473956a9-6e1e-4b1e-be4a-78da68ad12c4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:11.088795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473956a9-6e1e-4b1e-be4a-78da68ad12c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:11.207620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-473956a9-6e1e-4b1e-be4a-78da68ad12c4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:11.210011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-473956a9-6e1e-4b1e-be4a-78da68ad12c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:11.371976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 665/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:11.684362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7e63-39a5-4987-af99-0ec9cdd6f282 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:11.684801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7e63-39a5-4987-af99-0ec9cdd6f282 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:11.685173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7e63-39a5-4987-af99-0ec9cdd6f282 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:11.686783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 644/1309] 10.222.0.15 () {54 vars in 965 bytes} [Fri Sep 9 14:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:11.693677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475905b1-6c51-4b0d-bd72-0d7c7227eed8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:11.693968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475905b1-6c51-4b0d-bd72-0d7c7227eed8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:11.694246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475905b1-6c51-4b0d-bd72-0d7c7227eed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:11.777611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-475905b1-6c51-4b0d-bd72-0d7c7227eed8 None None] MFA Rules not processed for user `f9edd8cf448e42fd8e888c8f531fa70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:11.778826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-475905b1-6c51-4b0d-bd72-0d7c7227eed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:11.801523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:11.879537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 666/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Fri Sep 9 14:49:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:11.888735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c7e63-39a5-4987-af99-0ec9cdd6f282 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:11.890616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3e0c7e63-39a5-4987-af99-0ec9cdd6f282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:12.033409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c7e63-39a5-4987-af99-0ec9cdd6f282 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:12.120464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463fb52-e2c9-499f-bec2-024bef299fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:12.121106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fb52-e2c9-499f-bec2-024bef299fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:12.121428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fb52-e2c9-499f-bec2-024bef299fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:12.121703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fb52-e2c9-499f-bec2-024bef299fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:12.123319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a463fb52-e2c9-499f-bec2-024bef299fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:12.183405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:12.244374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463fb52-e2c9-499f-bec2-024bef299fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:12.245764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463fb52-e2c9-499f-bec2-024bef299fe6 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:12.267096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 645/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:14.741063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b2bcc-6bf1-467d-839e-08cc8ae648a7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:14.741601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b2bcc-6bf1-467d-839e-08cc8ae648a7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:14.741735 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b2bcc-6bf1-467d-839e-08cc8ae648a7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:14.742993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 667/1312] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:14.748274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bf1bd-61c3-4e54-b52e-c45b6d5bd24a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:14.748709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bf1bd-61c3-4e54-b52e-c45b6d5bd24a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:14.748812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bf1bd-61c3-4e54-b52e-c45b6d5bd24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:14.838257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-751bf1bd-61c3-4e54-b52e-c45b6d5bd24a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:14.840181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-751bf1bd-61c3-4e54-b52e-c45b6d5bd24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:14.936496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 646/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:15.215209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b2bcc-6bf1-467d-839e-08cc8ae648a7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:15.216749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-859b2bcc-6bf1-467d-839e-08cc8ae648a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:15.317794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b2bcc-6bf1-467d-839e-08cc8ae648a7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:15.389631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:15.392560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42558870-479a-4007-aebd-4a72bf376c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:15.393397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42558870-479a-4007-aebd-4a72bf376c16 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:15.393718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42558870-479a-4007-aebd-4a72bf376c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:15.394187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42558870-479a-4007-aebd-4a72bf376c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:15.396366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-42558870-479a-4007-aebd-4a72bf376c16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:15.503351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42558870-479a-4007-aebd-4a72bf376c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:15.504713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42558870-479a-4007-aebd-4a72bf376c16 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:15.523100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 668/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:17.957359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ff02f-95cf-47f9-a39d-049604e21792 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:17.957920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ff02f-95cf-47f9-a39d-049604e21792 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:17.957920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ff02f-95cf-47f9-a39d-049604e21792 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:17.959251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 647/1315] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:17.964472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b02f6-0ee7-4132-b568-2763999c0767 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:17.964862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b02f6-0ee7-4132-b568-2763999c0767 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:17.965014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b02f6-0ee7-4132-b568-2763999c0767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:18.031733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e41b02f6-0ee7-4132-b568-2763999c0767 None None] MFA Rules not processed for user `55f51cbb4c144d569e357010b10f0ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:18.032953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e41b02f6-0ee7-4132-b568-2763999c0767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:18.084034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:18.126792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 669/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:18.406831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ff02f-95cf-47f9-a39d-049604e21792 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:18.408403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0b2ff02f-95cf-47f9-a39d-049604e21792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:18.419070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:18.511124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ff02f-95cf-47f9-a39d-049604e21792 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:18.584518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fa5ca-7b83-4636-bc1e-ba7833aa3a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:18.585113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fa5ca-7b83-4636-bc1e-ba7833aa3a3a service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:18.585414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fa5ca-7b83-4636-bc1e-ba7833aa3a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:18.585680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fa5ca-7b83-4636-bc1e-ba7833aa3a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:18.587283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7f0fa5ca-7b83-4636-bc1e-ba7833aa3a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:18.647914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:18.693409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fa5ca-7b83-4636-bc1e-ba7833aa3a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:18.694819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fa5ca-7b83-4636-bc1e-ba7833aa3a3a service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:18.713212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 648/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:21.372825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245aacf1-00ff-4759-b2d0-57fc7c6dcb2e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:21.373377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245aacf1-00ff-4759-b2d0-57fc7c6dcb2e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:21.373377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245aacf1-00ff-4759-b2d0-57fc7c6dcb2e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:21.375275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 670/1318] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:21.380433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f716da-e2c4-4d20-b188-f2862311a403 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:21.380802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f716da-e2c4-4d20-b188-f2862311a403 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:21.380984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f716da-e2c4-4d20-b188-f2862311a403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:21.445179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-52f716da-e2c4-4d20-b188-f2862311a403 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:21.446401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-52f716da-e2c4-4d20-b188-f2862311a403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:21.542804 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 649/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:21.824520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245aacf1-00ff-4759-b2d0-57fc7c6dcb2e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:21.826079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-245aacf1-00ff-4759-b2d0-57fc7c6dcb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:21.926756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245aacf1-00ff-4759-b2d0-57fc7c6dcb2e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:21.997646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcc9d4-02ca-4353-aeac-93a12b397494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:21.998238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcc9d4-02ca-4353-aeac-93a12b397494 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:21.998516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcc9d4-02ca-4353-aeac-93a12b397494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:21.998786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcc9d4-02ca-4353-aeac-93a12b397494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:22.000398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dddcc9d4-02ca-4353-aeac-93a12b397494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:22.007080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:22.108445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcc9d4-02ca-4353-aeac-93a12b397494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:22.109870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcc9d4-02ca-4353-aeac-93a12b397494 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:22.127758 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 671/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:24.912498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1f3a7-c3f9-4585-bd83-2c96113dba09 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:24.913001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1f3a7-c3f9-4585-bd83-2c96113dba09 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:24.913001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1f3a7-c3f9-4585-bd83-2c96113dba09 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:24.914365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 650/1321] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:24.919774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b6cc3-d580-4509-920f-88d7e341fb79 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:24.920190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b6cc3-d580-4509-920f-88d7e341fb79 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:24.920461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b6cc3-d580-4509-920f-88d7e341fb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:24.993931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cf8b6cc3-d580-4509-920f-88d7e341fb79 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:24.995177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cf8b6cc3-d580-4509-920f-88d7e341fb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:25.061487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:25.103437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 672/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:25.388498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1f3a7-c3f9-4585-bd83-2c96113dba09 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:25.390749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7de1f3a7-c3f9-4585-bd83-2c96113dba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:25.410923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:25.497998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1f3a7-c3f9-4585-bd83-2c96113dba09 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:25.573549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af930d-a25a-49ac-a026-2129e25328db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:25.574188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af930d-a25a-49ac-a026-2129e25328db service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:25.574466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af930d-a25a-49ac-a026-2129e25328db service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:25.574763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af930d-a25a-49ac-a026-2129e25328db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:25.576423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e9af930d-a25a-49ac-a026-2129e25328db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:25.648734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:25.687812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af930d-a25a-49ac-a026-2129e25328db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:25.689266 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af930d-a25a-49ac-a026-2129e25328db service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:25.706901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 651/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:27.905398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea2c98-b585-4967-9981-f302aadc6678 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:27.906458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea2c98-b585-4967-9981-f302aadc6678 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:27.906562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea2c98-b585-4967-9981-f302aadc6678 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:27.907924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 673/1324] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:27.913228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26878b46-c2e1-49dd-9f0a-48e1770f449d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:27.913627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26878b46-c2e1-49dd-9f0a-48e1770f449d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:27.913890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26878b46-c2e1-49dd-9f0a-48e1770f449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:27.986570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-26878b46-c2e1-49dd-9f0a-48e1770f449d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:27.987781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-26878b46-c2e1-49dd-9f0a-48e1770f449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:28.091525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 652/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:28.386483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26467b-2853-4e93-a3c4-e141ac343257 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:28.386764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26467b-2853-4e93-a3c4-e141ac343257 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:28.387039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26467b-2853-4e93-a3c4-e141ac343257 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:28.388662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 674/1326] 10.222.0.15 () {54 vars in 965 bytes} [Fri Sep 9 14:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:28.395162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ad226-3cac-4733-a273-e9292179ec95 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:28.395454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ad226-3cac-4733-a273-e9292179ec95 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:28.395729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ad226-3cac-4733-a273-e9292179ec95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:28.426542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:28.473370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4d9ad226-3cac-4733-a273-e9292179ec95 None None] MFA Rules not processed for user `f9edd8cf448e42fd8e888c8f531fa70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:28.474576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4d9ad226-3cac-4733-a273-e9292179ec95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:28.557483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 653/1327] 10.222.0.15 () {58 vars in 1062 bytes} [Fri Sep 9 14:49:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:28.567356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26467b-2853-4e93-a3c4-e141ac343257 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:28.568888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6d26467b-2853-4e93-a3c4-e141ac343257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:28.672277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26467b-2853-4e93-a3c4-e141ac343257 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:28.744503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a2243-15dc-49b1-a9d9-2a477d6a187e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:28.745107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2243-15dc-49b1-a9d9-2a477d6a187e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:28.745413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2243-15dc-49b1-a9d9-2a477d6a187e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:28.745685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2243-15dc-49b1-a9d9-2a477d6a187e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:28.747286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cc8a2243-15dc-49b1-a9d9-2a477d6a187e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:28.765474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:28.857287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a2243-15dc-49b1-a9d9-2a477d6a187e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:28.859514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a2243-15dc-49b1-a9d9-2a477d6a187e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:28.881679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 675/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:31.320437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4125c-e651-424e-b07e-c8d77043ea29 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:31.320983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4125c-e651-424e-b07e-c8d77043ea29 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:31.320983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4125c-e651-424e-b07e-c8d77043ea29 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:31.322361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 654/1329] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:31.327743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0111d-768d-4382-8f57-48ba411f3520 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:31.328227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0111d-768d-4382-8f57-48ba411f3520 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:31.328315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0111d-768d-4382-8f57-48ba411f3520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:31.398841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dca0111d-768d-4382-8f57-48ba411f3520 None None] MFA Rules not processed for user `55f51cbb4c144d569e357010b10f0ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:31.400147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dca0111d-768d-4382-8f57-48ba411f3520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:31.467602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:31.501904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 676/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:31.780614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4125c-e651-424e-b07e-c8d77043ea29 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:31.782198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fdb4125c-e651-424e-b07e-c8d77043ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:31.886216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4125c-e651-424e-b07e-c8d77043ea29 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:31.906830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:31.964230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e51231-3144-4114-bce8-5141bb74c85c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:31.964830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51231-3144-4114-bce8-5141bb74c85c service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:31.965105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51231-3144-4114-bce8-5141bb74c85c service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:31.965401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51231-3144-4114-bce8-5141bb74c85c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:31.966984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-40e51231-3144-4114-bce8-5141bb74c85c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:32.085298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e51231-3144-4114-bce8-5141bb74c85c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:32.086764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e51231-3144-4114-bce8-5141bb74c85c service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:32.105436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 655/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:34.610529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e586e-f46c-4ee9-b979-df9498ded711 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:34.611061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e586e-f46c-4ee9-b979-df9498ded711 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:34.611156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e586e-f46c-4ee9-b979-df9498ded711 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:34.612432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 677/1332] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:34.617709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28b939-68cd-4ead-9187-18fae7a19612 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:34.618133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28b939-68cd-4ead-9187-18fae7a19612 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:34.618253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28b939-68cd-4ead-9187-18fae7a19612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:34.680949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:34.683789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8d28b939-68cd-4ead-9187-18fae7a19612 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:34.685541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8d28b939-68cd-4ead-9187-18fae7a19612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:34.779739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 656/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:35.062228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e586e-f46c-4ee9-b979-df9498ded711 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:35.063764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2f2e586e-f46c-4ee9-b979-df9498ded711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:35.167821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e586e-f46c-4ee9-b979-df9498ded711 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:35.239046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafb0b8-41fb-41ee-9023-82c39d6cda6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:35.239638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafb0b8-41fb-41ee-9023-82c39d6cda6f service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:35.239914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafb0b8-41fb-41ee-9023-82c39d6cda6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:35.240183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafb0b8-41fb-41ee-9023-82c39d6cda6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:35.241794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8eafb0b8-41fb-41ee-9023-82c39d6cda6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:35.268539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:35.349230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafb0b8-41fb-41ee-9023-82c39d6cda6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:35.350680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafb0b8-41fb-41ee-9023-82c39d6cda6f service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:35.368437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 678/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:37.544347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a1618-2953-40c5-9d59-7e5ca624c4cb None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:37.544912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a1618-2953-40c5-9d59-7e5ca624c4cb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:37.544912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a1618-2953-40c5-9d59-7e5ca624c4cb None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:37.546785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 657/1335] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:37.552038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c87b31-719b-4920-b536-b792c569cef3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:37.552441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c87b31-719b-4920-b536-b792c569cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:37.552581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c87b31-719b-4920-b536-b792c569cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:37.621971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-36c87b31-719b-4920-b536-b792c569cef3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:37.623180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-36c87b31-719b-4920-b536-b792c569cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:37.696656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:37.718631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 679/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:38.005691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a1618-2953-40c5-9d59-7e5ca624c4cb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:38.007243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-749a1618-2953-40c5-9d59-7e5ca624c4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:38.109627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a1618-2953-40c5-9d59-7e5ca624c4cb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:38.139437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:38.184712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6b222-3330-4140-9355-952c8f313634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:38.185348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6b222-3330-4140-9355-952c8f313634 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:38.185623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6b222-3330-4140-9355-952c8f313634 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:38.185889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6b222-3330-4140-9355-952c8f313634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:38.187549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-12d6b222-3330-4140-9355-952c8f313634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:38.292224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6b222-3330-4140-9355-952c8f313634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:38.293614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6b222-3330-4140-9355-952c8f313634 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:38.310845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 658/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:40.898151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e73fc5-018d-4d93-9a30-a304fb162d76 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:40.898679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e73fc5-018d-4d93-9a30-a304fb162d76 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:40.898679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e73fc5-018d-4d93-9a30-a304fb162d76 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:40.900034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 680/1338] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:40.905749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe826285-33c7-4d91-8a57-570bf4c5f92a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:40.906150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe826285-33c7-4d91-8a57-570bf4c5f92a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:40.906298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe826285-33c7-4d91-8a57-570bf4c5f92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:40.973180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-fe826285-33c7-4d91-8a57-570bf4c5f92a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:40.974339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fe826285-33c7-4d91-8a57-570bf4c5f92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:40.982549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:41.070575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 659/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:41.356485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e73fc5-018d-4d93-9a30-a304fb162d76 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:41.358143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-73e73fc5-018d-4d93-9a30-a304fb162d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:41.459969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e73fc5-018d-4d93-9a30-a304fb162d76 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:41.533589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc7bae-df00-4a29-989b-b611f5626849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:41.534116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7bae-df00-4a29-989b-b611f5626849 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:41.534395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7bae-df00-4a29-989b-b611f5626849 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:41.534666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7bae-df00-4a29-989b-b611f5626849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:41.536259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-33fc7bae-df00-4a29-989b-b611f5626849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:41.570188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:41.643320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc7bae-df00-4a29-989b-b611f5626849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:41.644774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc7bae-df00-4a29-989b-b611f5626849 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:41.669734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 681/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:45.809379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4bfd4-7bbf-48cc-8514-b2950cc6a121 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:45.809985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4bfd4-7bbf-48cc-8514-b2950cc6a121 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:45.809985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4bfd4-7bbf-48cc-8514-b2950cc6a121 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:45.811906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 660/1341] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:45.817075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cd1cb-ef85-48ac-b3cd-3e71f3f77f2e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:45.817556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cd1cb-ef85-48ac-b3cd-3e71f3f77f2e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:45.817662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cd1cb-ef85-48ac-b3cd-3e71f3f77f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:45.884220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a56cd1cb-ef85-48ac-b3cd-3e71f3f77f2e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:45.885452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a56cd1cb-ef85-48ac-b3cd-3e71f3f77f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:45.971745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:45.983865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 682/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:46.267039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4bfd4-7bbf-48cc-8514-b2950cc6a121 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:46.268660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d4c4bfd4-7bbf-48cc-8514-b2950cc6a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:46.379590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4bfd4-7bbf-48cc-8514-b2950cc6a121 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:46.418563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:46.457251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13c32-4f5d-4ef1-b112-db2e9bf77153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:46.457861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13c32-4f5d-4ef1-b112-db2e9bf77153 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:46.458142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13c32-4f5d-4ef1-b112-db2e9bf77153 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:46.458416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13c32-4f5d-4ef1-b112-db2e9bf77153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:46.460031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1cd13c32-4f5d-4ef1-b112-db2e9bf77153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:46.570841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13c32-4f5d-4ef1-b112-db2e9bf77153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:46.572211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13c32-4f5d-4ef1-b112-db2e9bf77153 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:46.590556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 661/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:49.411475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37915b-c957-4598-bacf-5515795dd071 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:49.412269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37915b-c957-4598-bacf-5515795dd071 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:49.412428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37915b-c957-4598-bacf-5515795dd071 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:49.414111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 683/1344] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:49:49.419401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396e36c-b3e2-4292-87d0-e1e4f3f9ce77 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:49.419822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396e36c-b3e2-4292-87d0-e1e4f3f9ce77 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:49.419933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396e36c-b3e2-4292-87d0-e1e4f3f9ce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:49.488205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7396e36c-b3e2-4292-87d0-e1e4f3f9ce77 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:49.489428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7396e36c-b3e2-4292-87d0-e1e4f3f9ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:49.509567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:49.598190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 662/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:49.885171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37915b-c957-4598-bacf-5515795dd071 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:49.886734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8b37915b-c957-4598-bacf-5515795dd071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:49.995116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37915b-c957-4598-bacf-5515795dd071 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:50.076956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288e9bf-b88c-4151-8b02-cc6bcc870cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:50.077584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288e9bf-b88c-4151-8b02-cc6bcc870cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:50.077860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288e9bf-b88c-4151-8b02-cc6bcc870cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:50.078129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288e9bf-b88c-4151-8b02-cc6bcc870cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:50.079719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1288e9bf-b88c-4151-8b02-cc6bcc870cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:50.122878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:50.191049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288e9bf-b88c-4151-8b02-cc6bcc870cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:50.192497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288e9bf-b88c-4151-8b02-cc6bcc870cc6 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:50.211507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 684/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:49:54.597505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b97566-5749-4307-b0e5-cb2fad038ad6 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:54.597992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b97566-5749-4307-b0e5-cb2fad038ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:54.598115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b97566-5749-4307-b0e5-cb2fad038ad6 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:54.599378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 663/1347] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:49:54.604624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac777ff0-b098-4d6d-8d49-8de6e5b3609c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:54.605002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac777ff0-b098-4d6d-8d49-8de6e5b3609c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:54.605174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac777ff0-b098-4d6d-8d49-8de6e5b3609c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:54.673599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ac777ff0-b098-4d6d-8d49-8de6e5b3609c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:49:54.674824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ac777ff0-b098-4d6d-8d49-8de6e5b3609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:54.797475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:54.799947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 685/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:49:55.082594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b97566-5749-4307-b0e5-cb2fad038ad6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:49:55.084165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b3b97566-5749-4307-b0e5-cb2fad038ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:55.185852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b97566-5749-4307-b0e5-cb2fad038ad6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:49:55.232813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:49:55.258020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c772-db70-4820-be6e-b64cdedfe095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:49:55.258658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c772-db70-4820-be6e-b64cdedfe095 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:49:55.258934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c772-db70-4820-be6e-b64cdedfe095 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:49:55.259202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c772-db70-4820-be6e-b64cdedfe095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:49:55.260802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9092c772-db70-4820-be6e-b64cdedfe095 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:49:55.368619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c772-db70-4820-be6e-b64cdedfe095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:49:55.370220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c772-db70-4820-be6e-b64cdedfe095 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:49:55.387800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 664/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:00.382781 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239c187-33ea-46f8-9466-739aa7c859d4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:00.383326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239c187-33ea-46f8-9466-739aa7c859d4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:00.383326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239c187-33ea-46f8-9466-739aa7c859d4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:00.384661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 686/1350] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:00.391950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b05b-ae61-4813-9ad9-b66399a40028 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:00.392404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b05b-ae61-4813-9ad9-b66399a40028 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:00.392500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b05b-ae61-4813-9ad9-b66399a40028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:00.490178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a8d3b05b-ae61-4813-9ad9-b66399a40028 None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:00.491444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a8d3b05b-ae61-4813-9ad9-b66399a40028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:00.511989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:00.613291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 665/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:04.629555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196840a4-dc13-41f2-b576-0de613235939 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:04.630108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196840a4-dc13-41f2-b576-0de613235939 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:04.630108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196840a4-dc13-41f2-b576-0de613235939 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:04.631667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 687/1352] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:04.636189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c250c4-be5c-4f13-ba5a-b63d31bd22b9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:04.636630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c250c4-be5c-4f13-ba5a-b63d31bd22b9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:04.636752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c250c4-be5c-4f13-ba5a-b63d31bd22b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:04.716621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-71c250c4-be5c-4f13-ba5a-b63d31bd22b9 None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:04.718382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-71c250c4-be5c-4f13-ba5a-b63d31bd22b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:04.793259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:04.833549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 666/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:06.297426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77a725-ba93-4e53-af3f-ded6d8c5275e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:06.297940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77a725-ba93-4e53-af3f-ded6d8c5275e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:06.297940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77a725-ba93-4e53-af3f-ded6d8c5275e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:06.299207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 688/1354] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:06.304216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116dded-90f6-4e20-99f0-73f4415d855a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:06.304647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116dded-90f6-4e20-99f0-73f4415d855a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:06.304815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116dded-90f6-4e20-99f0-73f4415d855a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:06.380955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6116dded-90f6-4e20-99f0-73f4415d855a None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:06.382185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6116dded-90f6-4e20-99f0-73f4415d855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:06.490466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 667/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:10.029545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3c188-4e5d-4541-8082-0a4d445adc72 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:10.030039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3c188-4e5d-4541-8082-0a4d445adc72 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:10.030138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3c188-4e5d-4541-8082-0a4d445adc72 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:10.035574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 689/1356] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:10.037671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dd4d8-0171-4b1a-ba6c-8a20658d5cff None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:10.039092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dd4d8-0171-4b1a-ba6c-8a20658d5cff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:10.039391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dd4d8-0171-4b1a-ba6c-8a20658d5cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:10.079482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:10.116434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-664dd4d8-0171-4b1a-ba6c-8a20658d5cff None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:10.118188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-664dd4d8-0171-4b1a-ba6c-8a20658d5cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:10.219479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 668/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:10.521830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75659-5497-4797-9f1d-0bd4c530d9d1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:10.522284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75659-5497-4797-9f1d-0bd4c530d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:10.522723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75659-5497-4797-9f1d-0bd4c530d9d1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:10.524589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 690/1358] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 14:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:10.533603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17bab-8942-4cb6-b0ae-8c48556d03a2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:10.534142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17bab-8942-4cb6-b0ae-8c48556d03a2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:10.534529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17bab-8942-4cb6-b0ae-8c48556d03a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:10.619629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9dc17bab-8942-4cb6-b0ae-8c48556d03a2 None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:10.620953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9dc17bab-8942-4cb6-b0ae-8c48556d03a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:10.673836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:10.748932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 669/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:10.756785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75659-5497-4797-9f1d-0bd4c530d9d1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:10.758391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7bf75659-5497-4797-9f1d-0bd4c530d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:10.918439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75659-5497-4797-9f1d-0bd4c530d9d1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:10.994070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:11.038503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b10cbf-920d-497f-83f2-bcd08744cf6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:11.039186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10cbf-920d-497f-83f2-bcd08744cf6a service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:11.039368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10cbf-920d-497f-83f2-bcd08744cf6a service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:11.039631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10cbf-920d-497f-83f2-bcd08744cf6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:11.041260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-45b10cbf-920d-497f-83f2-bcd08744cf6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:11.139505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b10cbf-920d-497f-83f2-bcd08744cf6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:11.141169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b10cbf-920d-497f-83f2-bcd08744cf6a service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:11.161743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 691/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:13.584977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2eef3-941f-48d6-b3fa-2652d16da80d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:13.585539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2eef3-941f-48d6-b3fa-2652d16da80d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:13.585622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2eef3-941f-48d6-b3fa-2652d16da80d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:13.587217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 670/1361] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:13.596113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4daa01-5db8-4bc8-ad60-3fbe86add897 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:13.596707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4daa01-5db8-4bc8-ad60-3fbe86add897 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:13.596976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4daa01-5db8-4bc8-ad60-3fbe86add897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:13.692753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6a4daa01-5db8-4bc8-ad60-3fbe86add897 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:13.694047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a4daa01-5db8-4bc8-ad60-3fbe86add897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:13.737360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:13.799726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 692/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:14.096258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2eef3-941f-48d6-b3fa-2652d16da80d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:14.098031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-efe2eef3-941f-48d6-b3fa-2652d16da80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:14.275704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2eef3-941f-48d6-b3fa-2652d16da80d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:14.287873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:14.435229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c875d4f-20b2-44e1-8d9f-da311cd79c45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:14.435854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c875d4f-20b2-44e1-8d9f-da311cd79c45 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:14.436258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c875d4f-20b2-44e1-8d9f-da311cd79c45 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:14.436528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c875d4f-20b2-44e1-8d9f-da311cd79c45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:14.438178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2c875d4f-20b2-44e1-8d9f-da311cd79c45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:14.530384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c875d4f-20b2-44e1-8d9f-da311cd79c45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:14.531897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c875d4f-20b2-44e1-8d9f-da311cd79c45 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:14.550172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 671/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:17.092740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcdb99-f330-4808-a506-db0bf74e47fd None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:17.093337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcdb99-f330-4808-a506-db0bf74e47fd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:17.093337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcdb99-f330-4808-a506-db0bf74e47fd None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:17.094654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 693/1364] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:17.100050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6232c0-afb5-4036-9b1a-6ec37f625f81 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:17.100809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6232c0-afb5-4036-9b1a-6ec37f625f81 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:17.101088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6232c0-afb5-4036-9b1a-6ec37f625f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:17.116559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:17.171127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bb6232c0-afb5-4036-9b1a-6ec37f625f81 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:17.172346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bb6232c0-afb5-4036-9b1a-6ec37f625f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:17.275339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 672/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:17.566732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d97a-018d-4ff4-9844-81223edd471f None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:17.567129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d97a-018d-4ff4-9844-81223edd471f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:17.567500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d97a-018d-4ff4-9844-81223edd471f None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:17.569120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 694/1366] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 14:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:17.574612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f79b6-3145-4d8b-bcd8-84e4ff593c31 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:17.574966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f79b6-3145-4d8b-bcd8-84e4ff593c31 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:17.575302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f79b6-3145-4d8b-bcd8-84e4ff593c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:17.655942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b48f79b6-3145-4d8b-bcd8-84e4ff593c31 None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:17.657248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b48f79b6-3145-4d8b-bcd8-84e4ff593c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:17.693838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:17.782666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 673/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:17.798088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d97a-018d-4ff4-9844-81223edd471f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:17.800333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6ed8d97a-018d-4ff4-9844-81223edd471f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:17.964467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d97a-018d-4ff4-9844-81223edd471f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:17.987080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:18.099360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff387d0-af4c-483f-995c-ce4e553613a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:18.100749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff387d0-af4c-483f-995c-ce4e553613a2 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:18.101223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff387d0-af4c-483f-995c-ce4e553613a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:18.101517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff387d0-af4c-483f-995c-ce4e553613a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:18.103283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2ff387d0-af4c-483f-995c-ce4e553613a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:18.200074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff387d0-af4c-483f-995c-ce4e553613a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:18.202230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff387d0-af4c-483f-995c-ce4e553613a2 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:18.219310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfee8fc-71f9-432a-85ed-e9e7b4e04de8 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:18.219838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfee8fc-71f9-432a-85ed-e9e7b4e04de8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:18.219838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfee8fc-71f9-432a-85ed-e9e7b4e04de8 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:18.222328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 695/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:18.225301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 674/1369] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:50:18.226791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff897c87-4f58-4b73-aa5b-f3113654270d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:18.227082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff897c87-4f58-4b73-aa5b-f3113654270d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:18.227360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff897c87-4f58-4b73-aa5b-f3113654270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:18.261108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:18.300611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ff897c87-4f58-4b73-aa5b-f3113654270d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:18.301873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ff897c87-4f58-4b73-aa5b-f3113654270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:18.420781 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 696/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:18.750840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0690e-7d8e-4260-8fdb-8f9ffbe8d070 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:18.750840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0690e-7d8e-4260-8fdb-8f9ffbe8d070 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:18.750840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0690e-7d8e-4260-8fdb-8f9ffbe8d070 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:18.753605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 675/1371] 10.222.0.15 () {54 vars in 964 bytes} [Fri Sep 9 14:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:18.757181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15237e2-2012-4e45-8b79-dbcb126ef0dc None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:18.757496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15237e2-2012-4e45-8b79-dbcb126ef0dc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:18.757767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15237e2-2012-4e45-8b79-dbcb126ef0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:18.842198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d15237e2-2012-4e45-8b79-dbcb126ef0dc None None] MFA Rules not processed for user `b46482f26051455daeed2c00f8ced4bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:18.843413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d15237e2-2012-4e45-8b79-dbcb126ef0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:18.893130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:18.927226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 697/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Fri Sep 9 14:50:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:18.934732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0690e-7d8e-4260-8fdb-8f9ffbe8d070 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:18.936380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-aeb0690e-7d8e-4260-8fdb-8f9ffbe8d070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:19.053567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0690e-7d8e-4260-8fdb-8f9ffbe8d070 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:19.098175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:19.135865 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6f9fd-30d5-42c6-840f-2295b36a5196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:19.136503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6f9fd-30d5-42c6-840f-2295b36a5196 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:19.136782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6f9fd-30d5-42c6-840f-2295b36a5196 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:19.137052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6f9fd-30d5-42c6-840f-2295b36a5196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:19.139721 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d5b6f9fd-30d5-42c6-840f-2295b36a5196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:19.289772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6f9fd-30d5-42c6-840f-2295b36a5196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:19.291433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6f9fd-30d5-42c6-840f-2295b36a5196 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:19.316801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 676/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:20.452122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9a9d-271c-447f-8198-c39e7d4b2dd0 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:20.452667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9a9d-271c-447f-8198-c39e7d4b2dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:20.452667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9a9d-271c-447f-8198-c39e7d4b2dd0 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:20.454027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 698/1374] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:20.459366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaeb58-42d4-46f1-97ab-b83a8e540cb8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:20.459791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaeb58-42d4-46f1-97ab-b83a8e540cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:20.460055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaeb58-42d4-46f1-97ab-b83a8e540cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:20.526206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d9eaeb58-42d4-46f1-97ab-b83a8e540cb8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:20.527420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d9eaeb58-42d4-46f1-97ab-b83a8e540cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:20.548302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:20.625017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 677/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:20.923555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb9a9d-271c-447f-8198-c39e7d4b2dd0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:20.925104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e9cb9a9d-271c-447f-8198-c39e7d4b2dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:21.086075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb9a9d-271c-447f-8198-c39e7d4b2dd0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:21.140379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:21.204886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f68b0-223c-40e2-88c1-b867db4758ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:21.205518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f68b0-223c-40e2-88c1-b867db4758ec service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:21.205790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f68b0-223c-40e2-88c1-b867db4758ec service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:21.206056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f68b0-223c-40e2-88c1-b867db4758ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:21.207632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ff6f68b0-223c-40e2-88c1-b867db4758ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:21.298964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f68b0-223c-40e2-88c1-b867db4758ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:21.300627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f68b0-223c-40e2-88c1-b867db4758ec service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:21.318899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 699/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:23.529386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99800be6-4b8b-4e8e-af2c-b1158394df8d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:23.529964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99800be6-4b8b-4e8e-af2c-b1158394df8d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:23.529964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99800be6-4b8b-4e8e-af2c-b1158394df8d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:23.531259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 678/1377] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:23.536379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390197c-fea5-4b04-8eed-94215c636abd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:23.536792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390197c-fea5-4b04-8eed-94215c636abd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:23.536926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390197c-fea5-4b04-8eed-94215c636abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:23.610563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f390197c-fea5-4b04-8eed-94215c636abd None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:23.611838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f390197c-fea5-4b04-8eed-94215c636abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:23.620197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:23.717232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 700/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:24.017768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99800be6-4b8b-4e8e-af2c-b1158394df8d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:24.019456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-99800be6-4b8b-4e8e-af2c-b1158394df8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:24.173699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99800be6-4b8b-4e8e-af2c-b1158394df8d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:24.177636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:24.294081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368bac0-3ba7-46c1-a073-4fb73b129126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:24.294629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368bac0-3ba7-46c1-a073-4fb73b129126 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:24.294908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368bac0-3ba7-46c1-a073-4fb73b129126 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:24.295180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368bac0-3ba7-46c1-a073-4fb73b129126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:24.296752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8368bac0-3ba7-46c1-a073-4fb73b129126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:24.392603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368bac0-3ba7-46c1-a073-4fb73b129126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:24.395710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368bac0-3ba7-46c1-a073-4fb73b129126 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:24.413727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 679/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:26.624045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba710f-9ec3-426f-8a41-aac9e6016a22 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:26.624511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba710f-9ec3-426f-8a41-aac9e6016a22 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:26.624589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba710f-9ec3-426f-8a41-aac9e6016a22 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:26.625986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 701/1380] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:26.631425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108a39-f148-438e-9c86-077aa8f65c67 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:26.631821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108a39-f148-438e-9c86-077aa8f65c67 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:26.631993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108a39-f148-438e-9c86-077aa8f65c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:26.646653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:26.713329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-81108a39-f148-438e-9c86-077aa8f65c67 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:26.714597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-81108a39-f148-438e-9c86-077aa8f65c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:26.829186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 680/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:27.118269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba710f-9ec3-426f-8a41-aac9e6016a22 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:27.119817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d4ba710f-9ec3-426f-8a41-aac9e6016a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:27.260613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:27.273561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba710f-9ec3-426f-8a41-aac9e6016a22 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:27.404584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85eeff-791a-4cb3-878b-0cd6c7b59013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:27.405187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85eeff-791a-4cb3-878b-0cd6c7b59013 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:27.405487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85eeff-791a-4cb3-878b-0cd6c7b59013 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:27.405750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85eeff-791a-4cb3-878b-0cd6c7b59013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:27.407347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ef85eeff-791a-4cb3-878b-0cd6c7b59013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:27.507174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85eeff-791a-4cb3-878b-0cd6c7b59013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:27.508888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85eeff-791a-4cb3-878b-0cd6c7b59013 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:27.519880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:27.529390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 702/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:29.709080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310d4dc-6cbb-4d6e-93c7-d49fdfb46a77 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:29.709636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310d4dc-6cbb-4d6e-93c7-d49fdfb46a77 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:29.709636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310d4dc-6cbb-4d6e-93c7-d49fdfb46a77 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:29.711051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 681/1383] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:29.717372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2414b91-7f39-4bb5-b69d-1533a7ff8168 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:29.717797 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2414b91-7f39-4bb5-b69d-1533a7ff8168 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:29.718130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2414b91-7f39-4bb5-b69d-1533a7ff8168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:29.796717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a2414b91-7f39-4bb5-b69d-1533a7ff8168 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:29.798315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a2414b91-7f39-4bb5-b69d-1533a7ff8168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:29.914999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 703/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:30.209330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310d4dc-6cbb-4d6e-93c7-d49fdfb46a77 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:30.210865 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6310d4dc-6cbb-4d6e-93c7-d49fdfb46a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:30.302900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:30.386713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310d4dc-6cbb-4d6e-93c7-d49fdfb46a77 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:30.527697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0d0db-1087-4eff-850a-f15c6a6e6d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:30.529000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0d0db-1087-4eff-850a-f15c6a6e6d43 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:30.529471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0d0db-1087-4eff-850a-f15c6a6e6d43 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:30.529808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0d0db-1087-4eff-850a-f15c6a6e6d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:30.531977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1ff0d0db-1087-4eff-850a-f15c6a6e6d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:30.584712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:30.625441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0d0db-1087-4eff-850a-f15c6a6e6d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:30.627080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0d0db-1087-4eff-850a-f15c6a6e6d43 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:30.645163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 682/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:33.197691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81398f3-1aec-41f1-bb6f-f01f5ecf628e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:33.198292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81398f3-1aec-41f1-bb6f-f01f5ecf628e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:33.198292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81398f3-1aec-41f1-bb6f-f01f5ecf628e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:33.199580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 704/1386] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:33.205338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e11aa-9af0-4074-be01-f930f623ff35 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:33.205756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e11aa-9af0-4074-be01-f930f623ff35 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:33.205890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e11aa-9af0-4074-be01-f930f623ff35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:33.274894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-425e11aa-9af0-4074-be01-f930f623ff35 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:33.276021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-425e11aa-9af0-4074-be01-f930f623ff35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:33.371016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 683/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:33.668930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81398f3-1aec-41f1-bb6f-f01f5ecf628e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:33.671151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d81398f3-1aec-41f1-bb6f-f01f5ecf628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:33.746354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:33.815304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81398f3-1aec-41f1-bb6f-f01f5ecf628e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:33.928918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775a9f8-6e2e-4f68-9a78-ceddfc8161f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:33.929485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775a9f8-6e2e-4f68-9a78-ceddfc8161f1 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:33.929772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775a9f8-6e2e-4f68-9a78-ceddfc8161f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:33.930029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775a9f8-6e2e-4f68-9a78-ceddfc8161f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:33.931629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a775a9f8-6e2e-4f68-9a78-ceddfc8161f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:33.971195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:34.022502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775a9f8-6e2e-4f68-9a78-ceddfc8161f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:34.025035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775a9f8-6e2e-4f68-9a78-ceddfc8161f1 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:34.045794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 705/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:34.552023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f657b-1347-440e-bed7-7b8561c05d97 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:34.552684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f657b-1347-440e-bed7-7b8561c05d97 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:34.552684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f657b-1347-440e-bed7-7b8561c05d97 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:34.553952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 684/1389] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:34.559156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd2a25-7817-47fc-a8f9-e7033a713218 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:34.559666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd2a25-7817-47fc-a8f9-e7033a713218 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:34.560337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd2a25-7817-47fc-a8f9-e7033a713218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:34.627547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7afd2a25-7817-47fc-a8f9-e7033a713218 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:34.628762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7afd2a25-7817-47fc-a8f9-e7033a713218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:34.725538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 706/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:35.032373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19883044-5442-4e25-99d1-62bcc3cd98ff None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:35.032663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19883044-5442-4e25-99d1-62bcc3cd98ff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:35.032943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19883044-5442-4e25-99d1-62bcc3cd98ff None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:35.034327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 685/1391] 10.222.0.15 () {54 vars in 966 bytes} [Fri Sep 9 14:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:35.040018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9751ba-54bc-42ba-accf-dfb089053587 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:35.040378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9751ba-54bc-42ba-accf-dfb089053587 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:35.040724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9751ba-54bc-42ba-accf-dfb089053587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:35.056630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:35.116584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1c9751ba-54bc-42ba-accf-dfb089053587 None None] MFA Rules not processed for user `04d1fc63b356499d80061e358fb447e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:35.117806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1c9751ba-54bc-42ba-accf-dfb089053587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:35.193686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 707/1392] 10.222.0.15 () {58 vars in 1063 bytes} [Fri Sep 9 14:50:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:35.203074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19883044-5442-4e25-99d1-62bcc3cd98ff None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:35.204897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-19883044-5442-4e25-99d1-62bcc3cd98ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:35.235942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:35.310371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19883044-5442-4e25-99d1-62bcc3cd98ff None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:35.383177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7087b-a1e7-4476-aa8f-e0d767e4af45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:35.383812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7087b-a1e7-4476-aa8f-e0d767e4af45 service glance] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:35.384090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7087b-a1e7-4476-aa8f-e0d767e4af45 service glance] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:35.384360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7087b-a1e7-4476-aa8f-e0d767e4af45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:35.386030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-45b7087b-a1e7-4476-aa8f-e0d767e4af45 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:35.462790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:35.496457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7087b-a1e7-4476-aa8f-e0d767e4af45 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:35.498849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7087b-a1e7-4476-aa8f-e0d767e4af45 service glance] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:35.519823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 686/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:36.271709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887607d-a799-47b2-85d8-015b63ffa807 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:36.272208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887607d-a799-47b2-85d8-015b63ffa807 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:36.272295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887607d-a799-47b2-85d8-015b63ffa807 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:36.272971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa5d22-d19b-4158-a471-01c64a5712b6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:36.273360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa5d22-d19b-4158-a471-01c64a5712b6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:36.273561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa5d22-d19b-4158-a471-01c64a5712b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:36.273669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 708/1394] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:36.279242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017f5a-63dd-4cd2-b99d-6a17cf6fe34e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:36.279524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017f5a-63dd-4cd2-b99d-6a17cf6fe34e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:36.279956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017f5a-63dd-4cd2-b99d-6a17cf6fe34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:36.342475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0afa5d22-d19b-4158-a471-01c64a5712b6 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:36.343693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0afa5d22-d19b-4158-a471-01c64a5712b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:36.347816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-68017f5a-63dd-4cd2-b99d-6a17cf6fe34e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:36.348981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-68017f5a-63dd-4cd2-b99d-6a17cf6fe34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:36.405024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:36.431103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 687/1395] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:36.447665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 709/1396] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:36.473719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f774f4-964c-440c-acb5-ce15abfcbc4c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:36.474089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f774f4-964c-440c-acb5-ce15abfcbc4c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:36.474366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f774f4-964c-440c-acb5-ce15abfcbc4c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:36.475743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 688/1397] 10.222.0.15 () {54 vars in 943 bytes} [Fri Sep 9 14:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:36.482057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:36.482336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:36.482607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:36.581428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] MFA Rules not processed for user `552b3ea5ceab4114908b8108d415c9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:36.583067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:36.693290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 710/1398] 10.222.0.15 () {58 vars in 1040 bytes} [Fri Sep 9 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:36.703048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f774f4-964c-440c-acb5-ce15abfcbc4c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:36.705079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-40f774f4-964c-440c-acb5-ce15abfcbc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:36.733391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:36.734594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:36.754636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:36.757607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:36.836222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe6f22-5c5c-452f-8e78-8faf993dc8cd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:36.842314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f774f4-964c-440c-acb5-ce15abfcbc4c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:36.951005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c24c58-e049-4a1a-ae2f-1da101d4757c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:36.951605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c24c58-e049-4a1a-ae2f-1da101d4757c service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:36.951878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c24c58-e049-4a1a-ae2f-1da101d4757c service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:36.952142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c24c58-e049-4a1a-ae2f-1da101d4757c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:36.953765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-05c24c58-e049-4a1a-ae2f-1da101d4757c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:36.960126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf138d-5d77-487d-adf6-7ec89671560a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:36.960717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf138d-5d77-487d-adf6-7ec89671560a service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:36.961051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf138d-5d77-487d-adf6-7ec89671560a service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:36.961351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf138d-5d77-487d-adf6-7ec89671560a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:36.962966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bfdf138d-5d77-487d-adf6-7ec89671560a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:36.985073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:36.996194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:37.050346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c24c58-e049-4a1a-ae2f-1da101d4757c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:37.052222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c24c58-e049-4a1a-ae2f-1da101d4757c service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:37.052757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf138d-5d77-487d-adf6-7ec89671560a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:37.054357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf138d-5d77-487d-adf6-7ec89671560a service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:37.063875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 689/1399] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 14:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 14:50:37.070759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 711/1400] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:39.284964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0f02c-78cb-413b-95b3-711dd58854fd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:39.285512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0f02c-78cb-413b-95b3-711dd58854fd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:39.285512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0f02c-78cb-413b-95b3-711dd58854fd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:39.286953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 690/1401] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:39.292427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1cc83-4547-4179-b305-812428775529 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:39.292844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1cc83-4547-4179-b305-812428775529 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:39.293032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1cc83-4547-4179-b305-812428775529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:39.368434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f3e1cc83-4547-4179-b305-812428775529 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:39.369719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f3e1cc83-4547-4179-b305-812428775529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:39.480117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 712/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:50:39.780248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0f02c-78cb-413b-95b3-711dd58854fd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:39.781847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f6b0f02c-78cb-413b-95b3-711dd58854fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:39.963001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0f02c-78cb-413b-95b3-711dd58854fd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:39.993802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:40.094090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b44b25-2d3f-45e7-bab9-a4f2e4c10181 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:40.094734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b44b25-2d3f-45e7-bab9-a4f2e4c10181 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:40.095009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b44b25-2d3f-45e7-bab9-a4f2e4c10181 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:40.095273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b44b25-2d3f-45e7-bab9-a4f2e4c10181 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:40.096893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-77b44b25-2d3f-45e7-bab9-a4f2e4c10181 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:40.193140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b44b25-2d3f-45e7-bab9-a4f2e4c10181 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:40.194899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b44b25-2d3f-45e7-bab9-a4f2e4c10181 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:40.214542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 691/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:42.425777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb97c00-97a0-4a28-94dc-5751402fdfc0 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:42.426364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb97c00-97a0-4a28-94dc-5751402fdfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:42.426364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb97c00-97a0-4a28-94dc-5751402fdfc0 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:42.427646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 713/1404] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:42.432801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc6bc7-ce8f-4468-9139-1bcd0c9e9573 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:42.433562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc6bc7-ce8f-4468-9139-1bcd0c9e9573 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:42.434020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc6bc7-ce8f-4468-9139-1bcd0c9e9573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:42.480949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:42.508398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4bfc6bc7-ce8f-4468-9139-1bcd0c9e9573 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:42.509728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4bfc6bc7-ce8f-4468-9139-1bcd0c9e9573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:42.621242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 692/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:42.911779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb97c00-97a0-4a28-94dc-5751402fdfc0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:42.913424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0cb97c00-97a0-4a28-94dc-5751402fdfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:42.924680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:43.108260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb97c00-97a0-4a28-94dc-5751402fdfc0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:43.198380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:43.233063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56f050-fd53-4e9a-b40f-97636700b9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:43.233644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56f050-fd53-4e9a-b40f-97636700b9c8 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:43.233919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56f050-fd53-4e9a-b40f-97636700b9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:43.234183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56f050-fd53-4e9a-b40f-97636700b9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:43.235769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5d56f050-fd53-4e9a-b40f-97636700b9c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:43.365122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56f050-fd53-4e9a-b40f-97636700b9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:43.366830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56f050-fd53-4e9a-b40f-97636700b9c8 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:43.386872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 714/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:45.691898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f9a64-5fa2-4f26-a5ed-3233bcd4705a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:45.692466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f9a64-5fa2-4f26-a5ed-3233bcd4705a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:45.693036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f9a64-5fa2-4f26-a5ed-3233bcd4705a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:45.695629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 693/1407] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:50:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:50:45.701263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93ff2-b26b-446f-af38-df305ad86520 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:45.701779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93ff2-b26b-446f-af38-df305ad86520 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:45.702211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93ff2-b26b-446f-af38-df305ad86520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:45.780354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2fd93ff2-b26b-446f-af38-df305ad86520 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:50:45.781726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2fd93ff2-b26b-446f-af38-df305ad86520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:45.824793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:45.890848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 715/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:50:46.197249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f9a64-5fa2-4f26-a5ed-3233bcd4705a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:50:46.198807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5e6f9a64-5fa2-4f26-a5ed-3233bcd4705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:46.314088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:46.370882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f9a64-5fa2-4f26-a5ed-3233bcd4705a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:50:46.505044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062fa48-4935-4c29-a24e-286d9d8041e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:50:46.505673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062fa48-4935-4c29-a24e-286d9d8041e0 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:50:46.505942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062fa48-4935-4c29-a24e-286d9d8041e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:50:46.506205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062fa48-4935-4c29-a24e-286d9d8041e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:50:46.507819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a062fa48-4935-4c29-a24e-286d9d8041e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:50:46.591595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:46.612930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062fa48-4935-4c29-a24e-286d9d8041e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:50:46.614772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062fa48-4935-4c29-a24e-286d9d8041e0 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:50:46.634375 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 694/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:50:55.889620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=24 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:50:56.632105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=13 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:00.218419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b0b33-ab6c-409a-bf60-1294cd1ca766 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:00.219035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b0b33-ab6c-409a-bf60-1294cd1ca766 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:00.219035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b0b33-ab6c-409a-bf60-1294cd1ca766 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:00.220315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 716/1410] 10.222.0.15 () {54 vars in 918 bytes} [Fri Sep 9 14:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:00.225502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c382752-c5bf-4bb9-8c44-45b6ca0a2705 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:00.225934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c382752-c5bf-4bb9-8c44-45b6ca0a2705 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:00.226076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c382752-c5bf-4bb9-8c44-45b6ca0a2705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:00.301510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5c382752-c5bf-4bb9-8c44-45b6ca0a2705 None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:00.302641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5c382752-c5bf-4bb9-8c44-45b6ca0a2705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:00.409710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 695/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:51:00.424328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f26e53-fff2-46e8-8697-4b2a69ed74d3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:00.424612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f26e53-fff2-46e8-8697-4b2a69ed74d3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:00.424901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f26e53-fff2-46e8-8697-4b2a69ed74d3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:00.426320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 717/1412] 10.222.0.15 () {54 vars in 918 bytes} [Fri Sep 9 14:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:00.430908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfceff-64be-4494-9b7e-717c1880c689 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:00.431192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfceff-64be-4494-9b7e-717c1880c689 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:00.431467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfceff-64be-4494-9b7e-717c1880c689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:00.503463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:00.506417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b6cfceff-64be-4494-9b7e-717c1880c689 None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:00.507926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b6cfceff-64be-4494-9b7e-717c1880c689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:00.620365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 696/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:03.235062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a13f-f31e-41e8-9248-adeb6e9f1189 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:03.235622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a13f-f31e-41e8-9248-adeb6e9f1189 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:03.235622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a13f-f31e-41e8-9248-adeb6e9f1189 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:03.236923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:03.243720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9dc8-cb22-43d0-84eb-ae1e62791275 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:03.244147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9dc8-cb22-43d0-84eb-ae1e62791275 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:03.244271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9dc8-cb22-43d0-84eb-ae1e62791275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:03.311753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-318d9dc8-cb22-43d0-84eb-ae1e62791275 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:03.312890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-318d9dc8-cb22-43d0-84eb-ae1e62791275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:03.366671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:03.408588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:03.697754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55a13f-f31e-41e8-9248-adeb6e9f1189 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:03.699344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9f55a13f-f31e-41e8-9248-adeb6e9f1189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:03.843285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55a13f-f31e-41e8-9248-adeb6e9f1189 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:03.914992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:03.954965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcea181-0b9c-4c92-be3d-348f9ca82dfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:03.955505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcea181-0b9c-4c92-be3d-348f9ca82dfa service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:03.955782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcea181-0b9c-4c92-be3d-348f9ca82dfa service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:03.956051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcea181-0b9c-4c92-be3d-348f9ca82dfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:03.957685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cfcea181-0b9c-4c92-be3d-348f9ca82dfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:04.045545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcea181-0b9c-4c92-be3d-348f9ca82dfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:04.047154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcea181-0b9c-4c92-be3d-348f9ca82dfa service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:04.065345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 719/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:05.199726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281c24-7ee2-4b65-8446-e356439f2699 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:05.200910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281c24-7ee2-4b65-8446-e356439f2699 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:05.201055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281c24-7ee2-4b65-8446-e356439f2699 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:05.202441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:05.208505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7c23-a737-47d3-a5d6-dbf88bbeafae None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:05.208963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7c23-a737-47d3-a5d6-dbf88bbeafae None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:05.209070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7c23-a737-47d3-a5d6-dbf88bbeafae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:05.285449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e26a7c23-a737-47d3-a5d6-dbf88bbeafae None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:05.286567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e26a7c23-a737-47d3-a5d6-dbf88bbeafae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:05.314179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:05.382708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:05.542611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281c24-7ee2-4b65-8446-e356439f2699 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:05.544559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-41281c24-7ee2-4b65-8446-e356439f2699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:05.687671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281c24-7ee2-4b65-8446-e356439f2699 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:05.727356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:05.801307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81980903-c7b6-4362-91d1-b087d59fa87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:05.801860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81980903-c7b6-4362-91d1-b087d59fa87a service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:05.802137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81980903-c7b6-4362-91d1-b087d59fa87a service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:05.802406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81980903-c7b6-4362-91d1-b087d59fa87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:05.803996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-81980903-c7b6-4362-91d1-b087d59fa87a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:05.891168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81980903-c7b6-4362-91d1-b087d59fa87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:05.892759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81980903-c7b6-4362-91d1-b087d59fa87a service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:05.910801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 699/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:07.035227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f83626-5611-49ca-92c2-32cdb9cb6c6c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:07.035660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f83626-5611-49ca-92c2-32cdb9cb6c6c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:07.035779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f83626-5611-49ca-92c2-32cdb9cb6c6c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:07.037125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:07.043455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686974b-745b-47e5-925b-bb98d13e9e75 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:07.044455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686974b-745b-47e5-925b-bb98d13e9e75 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:07.044586 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686974b-745b-47e5-925b-bb98d13e9e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:07.097791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:07.120193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a686974b-745b-47e5-925b-bb98d13e9e75 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:07.121429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a686974b-745b-47e5-925b-bb98d13e9e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:07.215933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:07.365928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f83626-5611-49ca-92c2-32cdb9cb6c6c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:07.367401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-45f83626-5611-49ca-92c2-32cdb9cb6c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:07.509186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f83626-5611-49ca-92c2-32cdb9cb6c6c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:07.521823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:07.624194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94e6a-a9a8-4d13-bbdd-3c73ad67ab5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:07.624795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94e6a-a9a8-4d13-bbdd-3c73ad67ab5e service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:07.625070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94e6a-a9a8-4d13-bbdd-3c73ad67ab5e service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:07.625382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94e6a-a9a8-4d13-bbdd-3c73ad67ab5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:07.626990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bda94e6a-a9a8-4d13-bbdd-3c73ad67ab5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:07.714034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94e6a-a9a8-4d13-bbdd-3c73ad67ab5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:07.715643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94e6a-a9a8-4d13-bbdd-3c73ad67ab5e service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:07.733893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 722/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:08.858407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5ebea-6404-43f2-91cb-e2b7647dba38 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:08.858944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5ebea-6404-43f2-91cb-e2b7647dba38 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:08.858944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5ebea-6404-43f2-91cb-e2b7647dba38 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:08.860323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:08.866766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee54cc-eee0-4372-bb60-4f6702a20730 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:08.867261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee54cc-eee0-4372-bb60-4f6702a20730 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:08.867360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee54cc-eee0-4372-bb60-4f6702a20730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:08.888705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:08.944298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bdee54cc-eee0-4372-bb60-4f6702a20730 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:08.945581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bdee54cc-eee0-4372-bb60-4f6702a20730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:09.048260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:09.199984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5ebea-6404-43f2-91cb-e2b7647dba38 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:09.201549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e4c5ebea-6404-43f2-91cb-e2b7647dba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:09.317174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:09.352988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5ebea-6404-43f2-91cb-e2b7647dba38 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:09.463060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad2a01-f322-4885-9135-6cbf95472a79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:09.463658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad2a01-f322-4885-9135-6cbf95472a79 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:09.463934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad2a01-f322-4885-9135-6cbf95472a79 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:09.464199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad2a01-f322-4885-9135-6cbf95472a79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:09.466315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-baad2a01-f322-4885-9135-6cbf95472a79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:09.541217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:09.555600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad2a01-f322-4885-9135-6cbf95472a79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:09.557191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad2a01-f322-4885-9135-6cbf95472a79 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:09.574922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 702/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:10.691650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be7e7-5966-4346-98b3-8dd0b6f2a12a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:10.692237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be7e7-5966-4346-98b3-8dd0b6f2a12a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:10.692237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be7e7-5966-4346-98b3-8dd0b6f2a12a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:10.693589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:10.699877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbcc8c-1123-4b5e-8813-19719e5beffd None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:10.700321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbcc8c-1123-4b5e-8813-19719e5beffd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:10.700433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbcc8c-1123-4b5e-8813-19719e5beffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:10.776492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-edcbcc8c-1123-4b5e-8813-19719e5beffd None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:10.777755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-edcbcc8c-1123-4b5e-8813-19719e5beffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:10.873768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:11.025684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be7e7-5966-4346-98b3-8dd0b6f2a12a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:11.027221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-765be7e7-5966-4346-98b3-8dd0b6f2a12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:11.110873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:11.171855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be7e7-5966-4346-98b3-8dd0b6f2a12a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:11.284802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309f7c-8074-4a49-ad19-7a40f0803aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:11.285395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309f7c-8074-4a49-ad19-7a40f0803aff service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:11.285675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309f7c-8074-4a49-ad19-7a40f0803aff service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:11.285943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309f7c-8074-4a49-ad19-7a40f0803aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:11.287550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aa309f7c-8074-4a49-ad19-7a40f0803aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:11.335888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:11.376399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa309f7c-8074-4a49-ad19-7a40f0803aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:11.378053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa309f7c-8074-4a49-ad19-7a40f0803aff service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:11.396641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 725/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:12.460770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808bcc4-7e2e-4b25-b405-73ffb8657d52 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:12.461357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808bcc4-7e2e-4b25-b405-73ffb8657d52 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:12.461357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808bcc4-7e2e-4b25-b405-73ffb8657d52 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:12.462784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:12.469716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d8c3-d2c6-48d2-8103-a58438db0707 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:12.470151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d8c3-d2c6-48d2-8103-a58438db0707 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:12.470324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d8c3-d2c6-48d2-8103-a58438db0707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:12.545715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6599d8c3-d2c6-48d2-8103-a58438db0707 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:12.546956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6599d8c3-d2c6-48d2-8103-a58438db0707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:12.641396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:12.790912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808bcc4-7e2e-4b25-b405-73ffb8657d52 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:12.792451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4808bcc4-7e2e-4b25-b405-73ffb8657d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:12.841780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:12.937864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808bcc4-7e2e-4b25-b405-73ffb8657d52 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:13.053503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da419151-270b-4f63-b746-d6c021936507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:13.054417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419151-270b-4f63-b746-d6c021936507 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:13.054695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419151-270b-4f63-b746-d6c021936507 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:13.054963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419151-270b-4f63-b746-d6c021936507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:13.056574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-da419151-270b-4f63-b746-d6c021936507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:13.063728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:13.146688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da419151-270b-4f63-b746-d6c021936507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:13.148282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da419151-270b-4f63-b746-d6c021936507 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:13.166486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 705/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:14.239744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774152b-d0a2-4b51-a13a-f4e9c19ba0c3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:14.240300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774152b-d0a2-4b51-a13a-f4e9c19ba0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:14.240300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774152b-d0a2-4b51-a13a-f4e9c19ba0c3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:14.241660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:14.247799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e165df-cc1c-4704-a29a-b10af4907da4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:14.248236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e165df-cc1c-4704-a29a-b10af4907da4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:14.248351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e165df-cc1c-4704-a29a-b10af4907da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:14.323248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-79e165df-cc1c-4704-a29a-b10af4907da4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:14.324375 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-79e165df-cc1c-4704-a29a-b10af4907da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:14.397776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:14.426744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:51:14.575960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774152b-d0a2-4b51-a13a-f4e9c19ba0c3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:14.577524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5774152b-d0a2-4b51-a13a-f4e9c19ba0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:14.595180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:14.722140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774152b-d0a2-4b51-a13a-f4e9c19ba0c3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:14.825574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:14.851040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb8302-1032-43f6-aa31-88fa45d90332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:14.851573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8302-1032-43f6-aa31-88fa45d90332 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:14.851846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8302-1032-43f6-aa31-88fa45d90332 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:14.852113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8302-1032-43f6-aa31-88fa45d90332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:14.853756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fbdb8302-1032-43f6-aa31-88fa45d90332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:14.940453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb8302-1032-43f6-aa31-88fa45d90332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:14.942083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb8302-1032-43f6-aa31-88fa45d90332 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:14.959949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 728/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:16.016258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5da3f-09d6-4bc9-b148-6f06baf73f29 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:16.016730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5da3f-09d6-4bc9-b148-6f06baf73f29 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:16.016819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5da3f-09d6-4bc9-b148-6f06baf73f29 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:16.018177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:16.024477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1990-a7d0-4ccf-94d9-8b0fe6fa538e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:16.024910 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1990-a7d0-4ccf-94d9-8b0fe6fa538e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:16.025033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1990-a7d0-4ccf-94d9-8b0fe6fa538e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:16.100612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-47cb1990-a7d0-4ccf-94d9-8b0fe6fa538e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:16.101853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-47cb1990-a7d0-4ccf-94d9-8b0fe6fa538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:16.142313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:16.195602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:51:16.343829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5da3f-09d6-4bc9-b148-6f06baf73f29 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:16.345382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f7c5da3f-09d6-4bc9-b148-6f06baf73f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:16.500602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5da3f-09d6-4bc9-b148-6f06baf73f29 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:16.554860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:16.612669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68684258-11f8-4035-8d81-2cfaf78c3c4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:16.613245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684258-11f8-4035-8d81-2cfaf78c3c4a service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:16.613522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684258-11f8-4035-8d81-2cfaf78c3c4a service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:16.613787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684258-11f8-4035-8d81-2cfaf78c3c4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:16.615365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-68684258-11f8-4035-8d81-2cfaf78c3c4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:16.703896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684258-11f8-4035-8d81-2cfaf78c3c4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:16.706506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684258-11f8-4035-8d81-2cfaf78c3c4a service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:16.726056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 708/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:17.772581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4e0e4-f3a2-4b74-8be4-707ba2a8c636 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:17.773151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4e0e4-f3a2-4b74-8be4-707ba2a8c636 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:17.773151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4e0e4-f3a2-4b74-8be4-707ba2a8c636 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:17.774656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:17.781550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115f44a-26ed-489e-940e-074cc4e9ec2a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:17.782019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115f44a-26ed-489e-940e-074cc4e9ec2a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:17.782206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115f44a-26ed-489e-940e-074cc4e9ec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:17.860602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0115f44a-26ed-489e-940e-074cc4e9ec2a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:17.861850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0115f44a-26ed-489e-940e-074cc4e9ec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:17.863839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:17.956591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:18.105398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4e0e4-f3a2-4b74-8be4-707ba2a8c636 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:18.106921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-03b4e0e4-f3a2-4b74-8be4-707ba2a8c636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:18.251034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4e0e4-f3a2-4b74-8be4-707ba2a8c636 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:18.278042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:18.364602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d1c4a-5b68-485d-9ea4-1ba1f926021d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:18.365121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1c4a-5b68-485d-9ea4-1ba1f926021d service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:18.365430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1c4a-5b68-485d-9ea4-1ba1f926021d service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:18.365699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1c4a-5b68-485d-9ea4-1ba1f926021d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:18.367316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a92d1c4a-5b68-485d-9ea4-1ba1f926021d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:18.457915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d1c4a-5b68-485d-9ea4-1ba1f926021d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:18.459527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d1c4a-5b68-485d-9ea4-1ba1f926021d service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:18.476965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 731/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:20.546197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd7859-8014-4802-a020-c6fe35f7eb2a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:20.546852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd7859-8014-4802-a020-c6fe35f7eb2a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:20.546852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd7859-8014-4802-a020-c6fe35f7eb2a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:20.548096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:20.553622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881208e-183b-4903-94ef-229c1899e510 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:20.554041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881208e-183b-4903-94ef-229c1899e510 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:20.554185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881208e-183b-4903-94ef-229c1899e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:20.594335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:20.619633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5881208e-183b-4903-94ef-229c1899e510 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:20.620795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5881208e-183b-4903-94ef-229c1899e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:20.714901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:21.000711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd7859-8014-4802-a020-c6fe35f7eb2a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:21.002303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a5dd7859-8014-4802-a020-c6fe35f7eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:21.106020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd7859-8014-4802-a020-c6fe35f7eb2a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:21.138054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:21.179764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a679d0-4298-44ba-8053-01453e53b18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:21.180372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a679d0-4298-44ba-8053-01453e53b18e service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:21.180652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a679d0-4298-44ba-8053-01453e53b18e service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:21.180925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a679d0-4298-44ba-8053-01453e53b18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:21.182555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-39a679d0-4298-44ba-8053-01453e53b18e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:21.289993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a679d0-4298-44ba-8053-01453e53b18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:21.291351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a679d0-4298-44ba-8053-01453e53b18e service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:21.309418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 711/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:23.608421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84db97a-78a1-4e1f-8a7f-17598d259f44 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:23.608997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84db97a-78a1-4e1f-8a7f-17598d259f44 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:23.608997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84db97a-78a1-4e1f-8a7f-17598d259f44 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:23.610330 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:23.615635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3f6bf-74e1-418d-af70-e57d9aa17d40 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:23.616047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3f6bf-74e1-418d-af70-e57d9aa17d40 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:23.616184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3f6bf-74e1-418d-af70-e57d9aa17d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:23.680836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-35b3f6bf-74e1-418d-af70-e57d9aa17d40 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:23.682052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-35b3f6bf-74e1-418d-af70-e57d9aa17d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:23.694906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:23.777894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:24.064099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84db97a-78a1-4e1f-8a7f-17598d259f44 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:24.065685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d84db97a-78a1-4e1f-8a7f-17598d259f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:24.166197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84db97a-78a1-4e1f-8a7f-17598d259f44 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:24.179939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:24.242138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4ced9-c330-4fb3-bb2e-da84fb778c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:24.242739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ced9-c330-4fb3-bb2e-da84fb778c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:24.243018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ced9-c330-4fb3-bb2e-da84fb778c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:24.243290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ced9-c330-4fb3-bb2e-da84fb778c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:24.244884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c8b4ced9-c330-4fb3-bb2e-da84fb778c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:24.472491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4ced9-c330-4fb3-bb2e-da84fb778c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:24.473947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4ced9-c330-4fb3-bb2e-da84fb778c3d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:24.491964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 734/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:26.992185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a692b0a-1f4f-4bda-9aa1-68ce99f08a3d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:26.992715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a692b0a-1f4f-4bda-9aa1-68ce99f08a3d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:26.992715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a692b0a-1f4f-4bda-9aa1-68ce99f08a3d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:26.994065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:26.999903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f85a5-38b2-4152-b3ff-abfc5e9aaafe None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:27.000347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f85a5-38b2-4152-b3ff-abfc5e9aaafe None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:27.000464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f85a5-38b2-4152-b3ff-abfc5e9aaafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:27.048719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:27.067403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-242f85a5-38b2-4152-b3ff-abfc5e9aaafe None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:27.068619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-242f85a5-38b2-4152-b3ff-abfc5e9aaafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:27.161084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:27.445451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a692b0a-1f4f-4bda-9aa1-68ce99f08a3d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:27.446999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0a692b0a-1f4f-4bda-9aa1-68ce99f08a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:27.549549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a692b0a-1f4f-4bda-9aa1-68ce99f08a3d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:27.588457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:27.622993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852686b-8011-4042-953d-b21601635d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:27.623595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852686b-8011-4042-953d-b21601635d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:27.623879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852686b-8011-4042-953d-b21601635d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:27.624153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852686b-8011-4042-953d-b21601635d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:27.625776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7852686b-8011-4042-953d-b21601635d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:27.733537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852686b-8011-4042-953d-b21601635d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:27.734874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852686b-8011-4042-953d-b21601635d22 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:27.752440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 714/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:30.019534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925704b9-6ad3-4c02-833f-5c70e19a4fbe None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:30.019911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925704b9-6ad3-4c02-833f-5c70e19a4fbe None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:30.020082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925704b9-6ad3-4c02-833f-5c70e19a4fbe None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:30.021495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:30.026775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbc60-ca5d-4461-8373-d2b26cbc08e6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:30.027215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbc60-ca5d-4461-8373-d2b26cbc08e6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:30.027379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbc60-ca5d-4461-8373-d2b26cbc08e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:30.092534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1fadbc60-ca5d-4461-8373-d2b26cbc08e6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:30.093765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1fadbc60-ca5d-4461-8373-d2b26cbc08e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:30.114417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:30.205889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:30.494216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925704b9-6ad3-4c02-833f-5c70e19a4fbe None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:30.495775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-925704b9-6ad3-4c02-833f-5c70e19a4fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:30.610293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925704b9-6ad3-4c02-833f-5c70e19a4fbe None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:30.631224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:30.683689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f2193-130f-40ff-b59d-9c2b6fe73a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:30.684279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f2193-130f-40ff-b59d-9c2b6fe73a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:30.684550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f2193-130f-40ff-b59d-9c2b6fe73a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:30.684816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f2193-130f-40ff-b59d-9c2b6fe73a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:30.686462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ec7f2193-130f-40ff-b59d-9c2b6fe73a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:30.793051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f2193-130f-40ff-b59d-9c2b6fe73a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:30.795103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f2193-130f-40ff-b59d-9c2b6fe73a3d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:30.812665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 737/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:33.048264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4692-8fe8-4e56-bfa3-f6cceb7e9078 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:33.048786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4692-8fe8-4e56-bfa3-f6cceb7e9078 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:33.048786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4692-8fe8-4e56-bfa3-f6cceb7e9078 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:33.050216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:33.055439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc671933-9855-4f38-ba7a-24592cd2924b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:33.055832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc671933-9855-4f38-ba7a-24592cd2924b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:33.055990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc671933-9855-4f38-ba7a-24592cd2924b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:33.119105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:33.122728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cc671933-9855-4f38-ba7a-24592cd2924b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:33.124484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cc671933-9855-4f38-ba7a-24592cd2924b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:33.218207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:33.506855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd4692-8fe8-4e56-bfa3-f6cceb7e9078 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:33.508397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-95cd4692-8fe8-4e56-bfa3-f6cceb7e9078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:33.726926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd4692-8fe8-4e56-bfa3-f6cceb7e9078 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:33.774461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:33.798779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4e6f8-07f8-4457-b789-093b1320598d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:33.799382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4e6f8-07f8-4457-b789-093b1320598d service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:33.799661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4e6f8-07f8-4457-b789-093b1320598d service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:33.799938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4e6f8-07f8-4457-b789-093b1320598d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:33.801563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a0e4e6f8-07f8-4457-b789-093b1320598d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:33.910356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4e6f8-07f8-4457-b789-093b1320598d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:33.912023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4e6f8-07f8-4457-b789-093b1320598d service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:33.929622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 717/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:36.142620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc37b45-729b-4566-be58-67f840a5441e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:36.143187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc37b45-729b-4566-be58-67f840a5441e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:36.143187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc37b45-729b-4566-be58-67f840a5441e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:36.144507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:36.149881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01edf18-852e-441e-b7b6-ce4f63af7a06 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:36.150294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01edf18-852e-441e-b7b6-ce4f63af7a06 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:36.150464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01edf18-852e-441e-b7b6-ce4f63af7a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:36.215329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b01edf18-852e-441e-b7b6-ce4f63af7a06 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:36.216624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b01edf18-852e-441e-b7b6-ce4f63af7a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:36.245062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:36.312094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:36.602154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc37b45-729b-4566-be58-67f840a5441e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:36.603702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3fc37b45-729b-4566-be58-67f840a5441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:36.705065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc37b45-729b-4566-be58-67f840a5441e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:36.734394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:36.779355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e65895-c4a2-4e6d-a25f-a10da372e68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:36.779950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65895-c4a2-4e6d-a25f-a10da372e68e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:36.780224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65895-c4a2-4e6d-a25f-a10da372e68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:36.780490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65895-c4a2-4e6d-a25f-a10da372e68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:36.782122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-70e65895-c4a2-4e6d-a25f-a10da372e68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:36.897870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e65895-c4a2-4e6d-a25f-a10da372e68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:36.899309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e65895-c4a2-4e6d-a25f-a10da372e68e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:36.917255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 740/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:39.153944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c0f10-b940-4a5c-9ba7-0796d0844df8 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:39.154501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c0f10-b940-4a5c-9ba7-0796d0844df8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:39.154501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c0f10-b940-4a5c-9ba7-0796d0844df8 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:39.155824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:39.161242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe73fb-befe-4f7d-9438-360fe128f581 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:39.161668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe73fb-befe-4f7d-9438-360fe128f581 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:39.161803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe73fb-befe-4f7d-9438-360fe128f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:39.225762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4cfe73fb-befe-4f7d-9438-360fe128f581 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:39.226953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4cfe73fb-befe-4f7d-9438-360fe128f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:39.235560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:39.322225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:39.607500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c0f10-b940-4a5c-9ba7-0796d0844df8 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:39.609029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2f0c0f10-b940-4a5c-9ba7-0796d0844df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:39.711385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c0f10-b940-4a5c-9ba7-0796d0844df8 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:39.766664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:39.784697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664b07c-4a6f-49fd-a865-455f9287997d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:39.785326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664b07c-4a6f-49fd-a865-455f9287997d service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:39.785620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664b07c-4a6f-49fd-a865-455f9287997d service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:39.785873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664b07c-4a6f-49fd-a865-455f9287997d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:39.787451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4664b07c-4a6f-49fd-a865-455f9287997d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:39.905293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664b07c-4a6f-49fd-a865-455f9287997d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:39.906722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664b07c-4a6f-49fd-a865-455f9287997d service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:39.924442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 720/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:42.128487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45e9a9-a431-4565-91a3-acb8f1b78193 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:42.129046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45e9a9-a431-4565-91a3-acb8f1b78193 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:42.129046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45e9a9-a431-4565-91a3-acb8f1b78193 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:42.130451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:42.135662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997809-7e72-4deb-9f7a-47deecf66cc9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:42.136092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997809-7e72-4deb-9f7a-47deecf66cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:42.136203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997809-7e72-4deb-9f7a-47deecf66cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:42.201966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f4997809-7e72-4deb-9f7a-47deecf66cc9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:42.203215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f4997809-7e72-4deb-9f7a-47deecf66cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:42.240902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:42.300668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:42.582817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45e9a9-a431-4565-91a3-acb8f1b78193 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:42.584378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aa45e9a9-a431-4565-91a3-acb8f1b78193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:42.685245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45e9a9-a431-4565-91a3-acb8f1b78193 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:42.722587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:42.759224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fe480-de36-40f6-8527-b9310b82ecf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:42.759860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe480-de36-40f6-8527-b9310b82ecf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:42.760136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe480-de36-40f6-8527-b9310b82ecf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:42.760405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe480-de36-40f6-8527-b9310b82ecf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:42.762031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-322fe480-de36-40f6-8527-b9310b82ecf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:42.869629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fe480-de36-40f6-8527-b9310b82ecf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:42.870984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fe480-de36-40f6-8527-b9310b82ecf6 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:42.889520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 743/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:45.054684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ed031-dead-4e70-847f-0c3af45e79a0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:45.055182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ed031-dead-4e70-847f-0c3af45e79a0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:45.055297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ed031-dead-4e70-847f-0c3af45e79a0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:45.056564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:45.064249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ce5ab-dd38-465e-bda3-927dcf3a9774 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:45.065040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ce5ab-dd38-465e-bda3-927dcf3a9774 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:45.065352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ce5ab-dd38-465e-bda3-927dcf3a9774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:45.129442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f17ce5ab-dd38-465e-bda3-927dcf3a9774 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:45.130705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f17ce5ab-dd38-465e-bda3-927dcf3a9774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:45.148690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:45.225918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:45.519989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ed031-dead-4e70-847f-0c3af45e79a0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:45.521544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-669ed031-dead-4e70-847f-0c3af45e79a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:45.626998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ed031-dead-4e70-847f-0c3af45e79a0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:45.700875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:45.707903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445c6d4-c5f4-4afd-b6a1-4dbaf0edd117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:45.708511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445c6d4-c5f4-4afd-b6a1-4dbaf0edd117 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:45.708792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445c6d4-c5f4-4afd-b6a1-4dbaf0edd117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:45.709067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445c6d4-c5f4-4afd-b6a1-4dbaf0edd117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:45.710695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1445c6d4-c5f4-4afd-b6a1-4dbaf0edd117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:45.817852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445c6d4-c5f4-4afd-b6a1-4dbaf0edd117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:45.819205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445c6d4-c5f4-4afd-b6a1-4dbaf0edd117 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:45.837311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 723/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:48.029985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67500890-1a59-49e9-896c-2d8e748449d7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:48.030453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67500890-1a59-49e9-896c-2d8e748449d7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:48.030569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67500890-1a59-49e9-896c-2d8e748449d7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:48.031877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:51:48.037277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0bf72-2571-419e-878f-5d7956de24f6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:48.037702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0bf72-2571-419e-878f-5d7956de24f6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:48.037825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0bf72-2571-419e-878f-5d7956de24f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:48.103837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-71a0bf72-2571-419e-878f-5d7956de24f6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:51:48.105069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-71a0bf72-2571-419e-878f-5d7956de24f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:48.152114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:48.199389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:51:48.483276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67500890-1a59-49e9-896c-2d8e748449d7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:51:48.484823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-67500890-1a59-49e9-896c-2d8e748449d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:48.588052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67500890-1a59-49e9-896c-2d8e748449d7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:51:48.635058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:48.661100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf2793-3486-4423-b1ad-8f0cdd3da3b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:51:48.661723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2793-3486-4423-b1ad-8f0cdd3da3b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:51:48.662003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2793-3486-4423-b1ad-8f0cdd3da3b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:51:48.662274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2793-3486-4423-b1ad-8f0cdd3da3b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:51:48.663878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ddf2793-3486-4423-b1ad-8f0cdd3da3b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:51:48.771914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf2793-3486-4423-b1ad-8f0cdd3da3b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:51:48.773387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf2793-3486-4423-b1ad-8f0cdd3da3b4 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:51:48.790892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 746/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:51:58.197888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=22 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:51:58.788661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=66 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:08.732837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeba9e3-85bc-467f-a282-fbde08898ae0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:08.733432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeba9e3-85bc-467f-a282-fbde08898ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:08.733432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeba9e3-85bc-467f-a282-fbde08898ae0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:08.734761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 725/1471] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:08.740015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0078e-676e-40c3-954d-45dbcb294524 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:08.740453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0078e-676e-40c3-954d-45dbcb294524 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:08.740565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0078e-676e-40c3-954d-45dbcb294524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:08.805343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-aee0078e-676e-40c3-954d-45dbcb294524 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:08.806597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aee0078e-676e-40c3-954d-45dbcb294524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:08.901264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 747/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:09.203411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ecb9-ac5d-47e7-be51-22606fe2bbde None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:09.203698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ecb9-ac5d-47e7-be51-22606fe2bbde None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:09.203969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ecb9-ac5d-47e7-be51-22606fe2bbde None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:09.205319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 726/1473] 10.222.0.15 () {54 vars in 966 bytes} [Fri Sep 9 14:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:09.210196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ac58b-ee05-4da9-aed1-8fef69e8e33b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:09.210479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ac58b-ee05-4da9-aed1-8fef69e8e33b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:09.210752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ac58b-ee05-4da9-aed1-8fef69e8e33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:09.287135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-409ac58b-ee05-4da9-aed1-8fef69e8e33b None None] MFA Rules not processed for user `04d1fc63b356499d80061e358fb447e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:09.288268 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-409ac58b-ee05-4da9-aed1-8fef69e8e33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:09.308208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:09.363785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 748/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Fri Sep 9 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:09.371538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725ecb9-ac5d-47e7-be51-22606fe2bbde None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:52:09.373061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4725ecb9-ac5d-47e7-be51-22606fe2bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:09.478212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725ecb9-ac5d-47e7-be51-22606fe2bbde None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:52:09.551922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4f990-c5c9-428e-90a5-941e9993e5eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:52:09.552458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4f990-c5c9-428e-90a5-941e9993e5eb service glance] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:09.552736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4f990-c5c9-428e-90a5-941e9993e5eb service glance] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:09.553007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4f990-c5c9-428e-90a5-941e9993e5eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:09.554628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-21c4f990-c5c9-428e-90a5-941e9993e5eb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:09.607850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:09.662264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4f990-c5c9-428e-90a5-941e9993e5eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:52:09.663685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4f990-c5c9-428e-90a5-941e9993e5eb service glance] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:52:09.682045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 727/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:52:11.926139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c696c-8f3d-46bb-944d-150a17d87006 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:11.926699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c696c-8f3d-46bb-944d-150a17d87006 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:11.926699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c696c-8f3d-46bb-944d-150a17d87006 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:11.928006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 749/1476] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:11.933339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b8125-ac52-41df-a434-427c0242957c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:11.933780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b8125-ac52-41df-a434-427c0242957c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:11.933914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b8125-ac52-41df-a434-427c0242957c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:11.999183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6b6b8125-ac52-41df-a434-427c0242957c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:12.000373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6b6b8125-ac52-41df-a434-427c0242957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:12.097645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 728/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:12.378215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c696c-8f3d-46bb-944d-150a17d87006 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:52:12.379757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-142c696c-8f3d-46bb-944d-150a17d87006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:12.501433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c696c-8f3d-46bb-944d-150a17d87006 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:52:12.550253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:12.593058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28624-6d60-403d-9ee8-b988c77a0c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:52:12.593675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28624-6d60-403d-9ee8-b988c77a0c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:12.593951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28624-6d60-403d-9ee8-b988c77a0c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:12.594216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28624-6d60-403d-9ee8-b988c77a0c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:12.594842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28624-6d60-403d-9ee8-b988c77a0c3f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:52:12.596560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-bcb28624-6d60-403d-9ee8-b988c77a0c3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:52:12.596560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:52:12.596560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:52:12.597719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28624-6d60-403d-9ee8-b988c77a0c3f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:52:12.616847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 750/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 14:52:12 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:52:14.794365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d572-47aa-4ac3-a14d-41e53f474fee None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:14.794834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d572-47aa-4ac3-a14d-41e53f474fee None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:14.794939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d572-47aa-4ac3-a14d-41e53f474fee None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:14.796278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 729/1479] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:14.801622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cd6b1-2cf9-447d-bd6e-712cd8a6f863 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:14.802003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cd6b1-2cf9-447d-bd6e-712cd8a6f863 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:14.802166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cd6b1-2cf9-447d-bd6e-712cd8a6f863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:14.867612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-214cd6b1-2cf9-447d-bd6e-712cd8a6f863 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:14.868813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-214cd6b1-2cf9-447d-bd6e-712cd8a6f863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:14.963460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 751/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:15.258465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0d572-47aa-4ac3-a14d-41e53f474fee None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:52:15.260155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-dad0d572-47aa-4ac3-a14d-41e53f474fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:15.270336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:15.405060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0d572-47aa-4ac3-a14d-41e53f474fee None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:52:15.482807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:15.516384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa6d48-4582-4e25-a1a3-dbf9063f26fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:52:15.516920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa6d48-4582-4e25-a1a3-dbf9063f26fc service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:15.517485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa6d48-4582-4e25-a1a3-dbf9063f26fc service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:15.517754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa6d48-4582-4e25-a1a3-dbf9063f26fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:15.519349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6ffa6d48-4582-4e25-a1a3-dbf9063f26fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:15.606915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa6d48-4582-4e25-a1a3-dbf9063f26fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:52:15.608477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa6d48-4582-4e25-a1a3-dbf9063f26fc service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:52:15.625702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 730/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:52:17.951456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f61e2-2ffd-4e21-b83c-e606711c57d1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:17.952020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f61e2-2ffd-4e21-b83c-e606711c57d1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:17.952020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f61e2-2ffd-4e21-b83c-e606711c57d1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:17.953467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 752/1482] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:17.959337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53721-a5ff-4fc9-b51e-0e554797c57b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:17.959755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53721-a5ff-4fc9-b51e-0e554797c57b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:17.959876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53721-a5ff-4fc9-b51e-0e554797c57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:18.026644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1df53721-a5ff-4fc9-b51e-0e554797c57b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:18.027849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1df53721-a5ff-4fc9-b51e-0e554797c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:18.073911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:18.123381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 731/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:18.412903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f61e2-2ffd-4e21-b83c-e606711c57d1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:52:18.414486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-282f61e2-2ffd-4e21-b83c-e606711c57d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:18.438307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:18.523558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f61e2-2ffd-4e21-b83c-e606711c57d1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:52:18.595947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391a3a-2466-4140-a3f4-bb427bfb85ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:52:18.596537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391a3a-2466-4140-a3f4-bb427bfb85ae service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:18.596808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391a3a-2466-4140-a3f4-bb427bfb85ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:18.597073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391a3a-2466-4140-a3f4-bb427bfb85ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:18.598680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a391a3a-2466-4140-a3f4-bb427bfb85ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:18.671215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:18.704662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391a3a-2466-4140-a3f4-bb427bfb85ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:52:18.706132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391a3a-2466-4140-a3f4-bb427bfb85ae service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:52:18.724045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 753/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:52:20.923383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9389111-18a1-4566-8a97-0195a467890d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:20.923945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9389111-18a1-4566-8a97-0195a467890d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:20.923945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9389111-18a1-4566-8a97-0195a467890d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:20.925281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 732/1485] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:20.930452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871916a-fccd-4b74-b110-62c2a20bd310 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:20.930874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871916a-fccd-4b74-b110-62c2a20bd310 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:20.930984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871916a-fccd-4b74-b110-62c2a20bd310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:20.996089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f871916a-fccd-4b74-b110-62c2a20bd310 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:20.997825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f871916a-fccd-4b74-b110-62c2a20bd310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:21.093812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 754/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:21.380534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9389111-18a1-4566-8a97-0195a467890d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:52:21.382109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a9389111-18a1-4566-8a97-0195a467890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:21.484718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9389111-18a1-4566-8a97-0195a467890d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:52:21.554413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:21.555988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b4df6-0e1c-43a7-a5b7-9dd64311cf38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:52:21.556730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b4df6-0e1c-43a7-a5b7-9dd64311cf38 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:21.557240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b4df6-0e1c-43a7-a5b7-9dd64311cf38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:21.557531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b4df6-0e1c-43a7-a5b7-9dd64311cf38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:21.559996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-800b4df6-0e1c-43a7-a5b7-9dd64311cf38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:21.666763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b4df6-0e1c-43a7-a5b7-9dd64311cf38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:52:21.668100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b4df6-0e1c-43a7-a5b7-9dd64311cf38 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:52:21.686057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 733/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:52:23.995345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcedbe7-a8f2-47a3-b063-80066262bef1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:23.995948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcedbe7-a8f2-47a3-b063-80066262bef1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:23.995948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcedbe7-a8f2-47a3-b063-80066262bef1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:23.997857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 755/1488] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:24.003321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a0a22-42b5-46d0-9749-1937d765a27d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:24.003790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a0a22-42b5-46d0-9749-1937d765a27d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:24.003897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a0a22-42b5-46d0-9749-1937d765a27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:24.070770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-606a0a22-42b5-46d0-9749-1937d765a27d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:24.071972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-606a0a22-42b5-46d0-9749-1937d765a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:24.124279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:24.167292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 734/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:24.463195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcedbe7-a8f2-47a3-b063-80066262bef1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:52:24.464771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dfcedbe7-a8f2-47a3-b063-80066262bef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:24.501012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:24.569239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcedbe7-a8f2-47a3-b063-80066262bef1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:52:24.642760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9f3e7-faec-4044-b5d1-0d9caf954602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:52:24.643351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f3e7-faec-4044-b5d1-0d9caf954602 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:24.643627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f3e7-faec-4044-b5d1-0d9caf954602 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:24.643896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f3e7-faec-4044-b5d1-0d9caf954602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:24.645513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b2a9f3e7-faec-4044-b5d1-0d9caf954602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:24.723481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:24.752478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9f3e7-faec-4044-b5d1-0d9caf954602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:52:24.753944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9f3e7-faec-4044-b5d1-0d9caf954602 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:52:24.772878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 756/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:52:27.126541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca582bb-bc8b-4717-acd1-87161cc4f2ba None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:27.127083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca582bb-bc8b-4717-acd1-87161cc4f2ba None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:27.127083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca582bb-bc8b-4717-acd1-87161cc4f2ba None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:27.128433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 735/1491] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 14:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:52:27.133909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ac91-a9fa-4b7b-a25e-bc1d335d9bf5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:27.134312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ac91-a9fa-4b7b-a25e-bc1d335d9bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:27.134482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ac91-a9fa-4b7b-a25e-bc1d335d9bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:27.199722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-11c7ac91-a9fa-4b7b-a25e-bc1d335d9bf5 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:52:27.200971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-11c7ac91-a9fa-4b7b-a25e-bc1d335d9bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:27.295206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 757/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:52:27.583176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca582bb-bc8b-4717-acd1-87161cc4f2ba None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:52:27.584707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8ca582bb-bc8b-4717-acd1-87161cc4f2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:27.686986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca582bb-bc8b-4717-acd1-87161cc4f2ba None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:52:27.766250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1e514-1874-4632-a64d-55fc90648fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:52:27.766856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1e514-1874-4632-a64d-55fc90648fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:52:27.767130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1e514-1874-4632-a64d-55fc90648fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:52:27.767396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1e514-1874-4632-a64d-55fc90648fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:52:27.769275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-12d1e514-1874-4632-a64d-55fc90648fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:52:27.777983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:27.878359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1e514-1874-4632-a64d-55fc90648fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:52:27.879781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1e514-1874-4632-a64d-55fc90648fa3 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:52:27.898117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 736/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:52:37.293150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=82 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:52:37.895793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=50 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:41.461493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:41.462081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:41.462081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:41.538038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:53:41.539178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:41.634918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 758/1494] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:53:41.642922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02de7-d78b-4a20-b188-cc4ef4a3b510 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:41.643398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02de7-d78b-4a20-b188-cc4ef4a3b510 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:41.643506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02de7-d78b-4a20-b188-cc4ef4a3b510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:41.718081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-09a02de7-d78b-4a20-b188-cc4ef4a3b510 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:53:41.719284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-09a02de7-d78b-4a20-b188-cc4ef4a3b510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:41.813884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 737/1495] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:53:41.818400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:41.819598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:41.830729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cec76-7bad-4e3d-a57e-fd8586cea061 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:41.897142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:41.930385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:41.930981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:41.931254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:41.931520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:41.932335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:53:41.932666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:41.933884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:41.933884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:53:41.933884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:53:41.934766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:41.946257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 759/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:53:41.954632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a02de7-d78b-4a20-b188-cc4ef4a3b510 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:41.963742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a02de7-d78b-4a20-b188-cc4ef4a3b510 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:42.025070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:42.059034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:42.059637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.059914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.060183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.060817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:42.062562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:42.062562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Sep 09 14:53:42.062562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: Sep 09 14:53:42.063747 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:42.119853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 738/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:53:41 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:53:42.124054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:42.125578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:42.159146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39c75-ac94-49d2-9129-1ab78eb01666 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:42.250726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:42.251320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.251595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.251864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.252491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:42.253552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:42.253552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:53:42.253552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:53:42.254748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:42.297117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 760/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:53:42 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:53:42.301375 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:42.302944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:42.364183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2babe-bfb7-478b-835f-4a183b4dfe4b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:42.452290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:42.463563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:42.464196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.464474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.464743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.465438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:42.466525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:42.466525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Sep 09 14:53:42.466525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: Sep 09 14:53:42.467415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:42.479769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 739/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:53:42.483948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:42.485495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:42.497233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a3f3b-f6fe-427b-9d54-17e7334034bf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:42.524133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:42.592136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17591cea-43a1-4615-8e3a-1ca135cf5140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:42.592737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591cea-43a1-4615-8e3a-1ca135cf5140 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.593013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591cea-43a1-4615-8e3a-1ca135cf5140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.593306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591cea-43a1-4615-8e3a-1ca135cf5140 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.593939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17591cea-43a1-4615-8e3a-1ca135cf5140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:42.594945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-17591cea-43a1-4615-8e3a-1ca135cf5140 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:42.594945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:53:42.594945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:53:42.595825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17591cea-43a1-4615-8e3a-1ca135cf5140 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:42.607969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 761/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:53:42.612150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:42.613699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:42.647184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa0d3d-bade-4233-8338-fadecb151e70 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:42.740019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f0057-424e-4b11-9bd0-536b61221c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:42.740592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f0057-424e-4b11-9bd0-536b61221c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.740868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f0057-424e-4b11-9bd0-536b61221c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.741136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f0057-424e-4b11-9bd0-536b61221c4c admin admin] PATH_INFO: `/v3/projects/1e5ba21875f94095a950dd4a8ad00d52/users/796ec30d5b084b0db548386080830cb0/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.764004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f0057-424e-4b11-9bd0-536b61221c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5ba21875f94095a950dd4a8ad00d52, user_id=796ec30d5b084b0db548386080830cb0, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:42.765024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-ea9f0057-424e-4b11-9bd0-536b61221c4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:42.765024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Sep 09 14:53:42.765024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: Sep 09 14:53:42.766984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f0057-424e-4b11-9bd0-536b61221c4c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:42.792026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 740/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:53:42 2022] PUT /identity/v3/projects/1e5ba21875f94095a950dd4a8ad00d52/users/796ec30d5b084b0db548386080830cb0/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:53:42.804026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cd7e9-b08c-4c32-a2b7-26440c211a98 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.804313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cd7e9-b08c-4c32-a2b7-26440c211a98 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.804588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cd7e9-b08c-4c32-a2b7-26440c211a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.864204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0b0cd7e9-b08c-4c32-a2b7-26440c211a98 None None] MFA Rules not processed for user `796ec30d5b084b0db548386080830cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:53:42.865702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0b0cd7e9-b08c-4c32-a2b7-26440c211a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:42.940955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 762/1502] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:53:42.954362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fe85c-1584-4e4f-b016-45a5eba4ff3d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.954647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fe85c-1584-4e4f-b016-45a5eba4ff3d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.954922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fe85c-1584-4e4f-b016-45a5eba4ff3d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.956250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 741/1503] 10.222.0.15 () {54 vars in 964 bytes} [Fri Sep 9 14:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:53:42.961113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf797cb6-d625-4877-914f-25ec5414c6be None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:42.961443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf797cb6-d625-4877-914f-25ec5414c6be None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:42.961718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf797cb6-d625-4877-914f-25ec5414c6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:42.992914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:43.041075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cf797cb6-d625-4877-914f-25ec5414c6be None None] MFA Rules not processed for user `b46482f26051455daeed2c00f8ced4bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:53:43.042400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cf797cb6-d625-4877-914f-25ec5414c6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:43.117891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 763/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Fri Sep 9 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:53:43.126487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fe85c-1584-4e4f-b016-45a5eba4ff3d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:43.127995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0d6fe85c-1584-4e4f-b016-45a5eba4ff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:43.177264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:43.231815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fe85c-1584-4e4f-b016-45a5eba4ff3d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:43.304411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:43.304939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:43.305232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:43.305500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:43.307078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:43.400977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:43.402414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:43.406329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:43.421398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 742/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:53:43.466102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a883-b116-444c-9f72-4c5aae8161ee None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:43.466385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a883-b116-444c-9f72-4c5aae8161ee None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:43.466658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a883-b116-444c-9f72-4c5aae8161ee None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:43.467937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 764/1506] 10.222.0.15 () {52 vars in 830 bytes} [Fri Sep 9 14:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:53:43.471456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:43.472646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:43.573128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d6d1-5859-4941-9412-f4b3eed85a19 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:43.665624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:43.666248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:43.666509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:43.666776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] PATH_INFO: `/v3/users/796ec30d5b084b0db548386080830cb0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:43.679284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796ec30d5b084b0db548386080830cb0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:43.680292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:43.680292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Sep 09 14:53:43.680292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: Sep 09 14:53:43.681534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:43.691008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:43.749671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796ec30d5b084b0db548386080830cb0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:43.771974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796ec30d5b084b0db548386080830cb0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:43.785948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796ec30d5b084b0db548386080830cb0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:43.795262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796ec30d5b084b0db548386080830cb0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:43.795699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796ec30d5b084b0db548386080830cb0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:43.801214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 743/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:53:43 2022] DELETE /identity/v3/users/796ec30d5b084b0db548386080830cb0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:53:43.805574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41576ade-568b-468b-a0a0-581703298f62 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:43.806045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41576ade-568b-468b-a0a0-581703298f62 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:43.806149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41576ade-568b-468b-a0a0-581703298f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:43.863059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-41576ade-568b-468b-a0a0-581703298f62 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:53:43.864186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-41576ade-568b-468b-a0a0-581703298f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:43.937709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:43.951182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 765/1508] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:53:43.960599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:43.961878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:44.075134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2db070-1e0c-4275-85e9-0099d9d74749 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:44.128071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:44.149026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78449bb-b96f-4b4b-a25c-018f25d3d8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:44.149660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78449bb-b96f-4b4b-a25c-018f25d3d8fc service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:44.149935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78449bb-b96f-4b4b-a25c-018f25d3d8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:44.150198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78449bb-b96f-4b4b-a25c-018f25d3d8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:44.151779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d78449bb-b96f-4b4b-a25c-018f25d3d8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:44.255628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78449bb-b96f-4b4b-a25c-018f25d3d8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:44.257034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78449bb-b96f-4b4b-a25c-018f25d3d8fc service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:44.275686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 744/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:53:44.603707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41576ade-568b-468b-a0a0-581703298f62 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:53:44.604971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-41576ade-568b-468b-a0a0-581703298f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:53:44.616712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41576ade-568b-468b-a0a0-581703298f62 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:53:44.715938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:53:44.716495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:53:44.716774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:53:44.717046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] PATH_INFO: `/v3/projects/1e5ba21875f94095a950dd4a8ad00d52` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:53:44.726715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5ba21875f94095a950dd4a8ad00d52)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:53:44.727746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:53:44.727746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:53:44.727746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:53:44.728949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:53:44.785802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5ba21875f94095a950dd4a8ad00d52', 'request_id': 'req-9a0b88b0-f384-406f-b120-1fefbf591651'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:44.799576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5ba21875f94095a950dd4a8ad00d52', 'request_id': 'req-9a0b88b0-f384-406f-b120-1fefbf591651'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:44.805784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5ba21875f94095a950dd4a8ad00d52', 'request_id': 'req-9a0b88b0-f384-406f-b120-1fefbf591651'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:44.830722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] The token cache is being invalidate because project 1e5ba21875f94095a950dd4a8ad00d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:53:44.831053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:44.831423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9a0b88b0-f384-406f-b120-1fefbf591651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:53:44.832914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 766/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:53:44 2022] DELETE /identity/v3/projects/1e5ba21875f94095a950dd4a8ad00d52 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:53:54.273546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=62 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:53:54.828529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:55:10.616255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b486af-e7b6-4614-b5e8-33a2edc5ee25 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:10.616858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b486af-e7b6-4614-b5e8-33a2edc5ee25 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:10.616858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b486af-e7b6-4614-b5e8-33a2edc5ee25 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:10.618190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:10.623083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb80dd-cac4-414f-970c-338a0c82dd8f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:10.623492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb80dd-cac4-414f-970c-338a0c82dd8f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:10.623639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb80dd-cac4-414f-970c-338a0c82dd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:10.703031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5ceb80dd-cac4-414f-970c-338a0c82dd8f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:10.704239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5ceb80dd-cac4-414f-970c-338a0c82dd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:10.806580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:10.962339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b486af-e7b6-4614-b5e8-33a2edc5ee25 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:10.964422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-91b486af-e7b6-4614-b5e8-33a2edc5ee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:11.091906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b486af-e7b6-4614-b5e8-33a2edc5ee25 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:11.097418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08219f5-a3f9-4ece-a09f-14a1b8b53231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:11.098019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08219f5-a3f9-4ece-a09f-14a1b8b53231 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:11.098298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08219f5-a3f9-4ece-a09f-14a1b8b53231 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:11.098601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08219f5-a3f9-4ece-a09f-14a1b8b53231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:11.106550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08219f5-a3f9-4ece-a09f-14a1b8b53231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:11.108120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08219f5-a3f9-4ece-a09f-14a1b8b53231 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:11.117385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 746/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:12.208155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44817-742f-4182-afe4-14247268e76e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:12.208719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44817-742f-4182-afe4-14247268e76e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:12.208719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44817-742f-4182-afe4-14247268e76e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:12.210090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:12.216879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33747d-4ecc-4f1c-b9af-ae64b224d413 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:12.217374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33747d-4ecc-4f1c-b9af-ae64b224d413 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:12.217482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33747d-4ecc-4f1c-b9af-ae64b224d413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:12.238137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1a33747d-4ecc-4f1c-b9af-ae64b224d413 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:12.239347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1a33747d-4ecc-4f1c-b9af-ae64b224d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:12.247631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:12.285811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad283b-2624-46d9-9998-c67b7919a065 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:12.286036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad283b-2624-46d9-9998-c67b7919a065 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:12.286308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad283b-2624-46d9-9998-c67b7919a065 None None] PATH_INFO: `/v3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:12.287556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 769/1516] 10.222.0.16 () {54 vars in 925 bytes} [Fri Sep 9 14:55:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:55:12.293278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f9ad-4653-4df3-ba42-a34e23285dbd None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:12.293570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f9ad-4653-4df3-ba42-a34e23285dbd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:12.293851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f9ad-4653-4df3-ba42-a34e23285dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:12.353093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8305f9ad-4653-4df3-ba42-a34e23285dbd None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:12.354325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8305f9ad-4653-4df3-ba42-a34e23285dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:12.398004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad283b-2624-46d9-9998-c67b7919a065 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:12.408928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:55:12.413374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad283b-2624-46d9-9998-c67b7919a065 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:12.418808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ca9df-bb4d-40d0-b362-d35566626208 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:12.419438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca9df-bb4d-40d0-b362-d35566626208 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:12.419716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca9df-bb4d-40d0-b362-d35566626208 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:12.419985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca9df-bb4d-40d0-b362-d35566626208 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:12.428144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 748/1517] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Sep 9 14:55:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:55:12.428294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ca9df-bb4d-40d0-b362-d35566626208 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:12.429794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ca9df-bb4d-40d0-b362-d35566626208 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:12.439934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 770/1518] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:55:15.928110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1c453-fc19-48a2-96e7-109310bb0f07 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:15.928544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1c453-fc19-48a2-96e7-109310bb0f07 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:15.928657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1c453-fc19-48a2-96e7-109310bb0f07 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:15.929999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 749/1519] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 9 14:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:15.935363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce31cc8-8b44-4f75-9458-bb7532806444 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:15.935805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce31cc8-8b44-4f75-9458-bb7532806444 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:15.935914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce31cc8-8b44-4f75-9458-bb7532806444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:15.956070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7ce31cc8-8b44-4f75-9458-bb7532806444 None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:15.957275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7ce31cc8-8b44-4f75-9458-bb7532806444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:15.965663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 771/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 9 14:55:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:15.973883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1c453-fc19-48a2-96e7-109310bb0f07 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:15.986753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1c453-fc19-48a2-96e7-109310bb0f07 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:15.991608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c14cee-7da8-4951-80c9-e8203285de56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:15.992171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c14cee-7da8-4951-80c9-e8203285de56 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:15.992445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c14cee-7da8-4951-80c9-e8203285de56 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:15.992711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c14cee-7da8-4951-80c9-e8203285de56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:16.000692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c14cee-7da8-4951-80c9-e8203285de56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:16.002284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c14cee-7da8-4951-80c9-e8203285de56 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:16.011853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 750/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:23.918574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070abb-f425-497d-9b57-327a05c8fa5b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:23.919176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070abb-f425-497d-9b57-327a05c8fa5b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:23.919296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070abb-f425-497d-9b57-327a05c8fa5b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:23.920595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:23.926845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e509-a4bf-4017-a1f7-1d2e5dae50e1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:23.927273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e509-a4bf-4017-a1f7-1d2e5dae50e1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:23.927382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e509-a4bf-4017-a1f7-1d2e5dae50e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:23.947238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4da6e509-a4bf-4017-a1f7-1d2e5dae50e1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:23.948427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4da6e509-a4bf-4017-a1f7-1d2e5dae50e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:23.956650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:24.110110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070abb-f425-497d-9b57-327a05c8fa5b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:24.125052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070abb-f425-497d-9b57-327a05c8fa5b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:24.130668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bfada-2cd9-4270-9b64-5051f4ade753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:24.131254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bfada-2cd9-4270-9b64-5051f4ade753 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:24.131537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bfada-2cd9-4270-9b64-5051f4ade753 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:24.131811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bfada-2cd9-4270-9b64-5051f4ade753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:24.140314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bfada-2cd9-4270-9b64-5051f4ade753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:24.141970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bfada-2cd9-4270-9b64-5051f4ade753 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:24.151467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 773/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:25.223220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbcf01-dd06-427e-bc6c-9c16a6430e8a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:25.223755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbcf01-dd06-427e-bc6c-9c16a6430e8a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:25.223755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbcf01-dd06-427e-bc6c-9c16a6430e8a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:25.225024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 14:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:25.232192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0748d9-fad2-4b03-9b24-951d7ef2b95e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:25.232689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0748d9-fad2-4b03-9b24-951d7ef2b95e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:25.233082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0748d9-fad2-4b03-9b24-951d7ef2b95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:25.255734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ec0748d9-fad2-4b03-9b24-951d7ef2b95e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:25.256938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ec0748d9-fad2-4b03-9b24-951d7ef2b95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:25.265005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 14:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:25.415109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbcf01-dd06-427e-bc6c-9c16a6430e8a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:25.429752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbcf01-dd06-427e-bc6c-9c16a6430e8a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:25.435182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b24099-5b9f-4c29-a574-03f8eced5a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:25.435765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b24099-5b9f-4c29-a574-03f8eced5a38 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:25.436043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b24099-5b9f-4c29-a574-03f8eced5a38 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:25.436312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b24099-5b9f-4c29-a574-03f8eced5a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:25.444190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b24099-5b9f-4c29-a574-03f8eced5a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:25.445780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b24099-5b9f-4c29-a574-03f8eced5a38 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:25.454977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 753/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:29.628953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed9d1c-deb9-4969-9355-21c4cfc50d5a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:29.629582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed9d1c-deb9-4969-9355-21c4cfc50d5a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:29.629582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed9d1c-deb9-4969-9355-21c4cfc50d5a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:29.630903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:29.636177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cabca-6e36-459f-acf1-dcc72b82bf73 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:29.636603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cabca-6e36-459f-acf1-dcc72b82bf73 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:29.636715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cabca-6e36-459f-acf1-dcc72b82bf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:29.657076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a42cabca-6e36-459f-acf1-dcc72b82bf73 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:29.658272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a42cabca-6e36-459f-acf1-dcc72b82bf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:29.665920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:29.953000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed9d1c-deb9-4969-9355-21c4cfc50d5a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:29.954715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-61ed9d1c-deb9-4969-9355-21c4cfc50d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:29.987322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:55:30.033728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed9d1c-deb9-4969-9355-21c4cfc50d5a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:30.037309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f057-882d-4f32-8bff-460f94f5815c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:30.037907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f057-882d-4f32-8bff-460f94f5815c service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:30.038180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f057-882d-4f32-8bff-460f94f5815c service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:30.038446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f057-882d-4f32-8bff-460f94f5815c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:30.047647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f057-882d-4f32-8bff-460f94f5815c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:30.049066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f057-882d-4f32-8bff-460f94f5815c service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:30.058210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 776/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:32.230707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71659b-3609-4a29-b0c8-aec9b28c8f20 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:32.231277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71659b-3609-4a29-b0c8-aec9b28c8f20 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:32.231277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71659b-3609-4a29-b0c8-aec9b28c8f20 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:32.233080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:55:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:32.238470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11389c-2b2a-49fd-8c0b-399b1e4877e1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:32.238877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11389c-2b2a-49fd-8c0b-399b1e4877e1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:32.239008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11389c-2b2a-49fd-8c0b-399b1e4877e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:32.258034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7f11389c-2b2a-49fd-8c0b-399b1e4877e1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:32.259214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7f11389c-2b2a-49fd-8c0b-399b1e4877e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:32.267430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:32.561919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71659b-3609-4a29-b0c8-aec9b28c8f20 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:32.572603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71659b-3609-4a29-b0c8-aec9b28c8f20 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:32.576617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8783a1-7d87-4e16-a321-1175d0c9f749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:32.577219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8783a1-7d87-4e16-a321-1175d0c9f749 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:32.577505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8783a1-7d87-4e16-a321-1175d0c9f749 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:32.577776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8783a1-7d87-4e16-a321-1175d0c9f749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:32.586537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8783a1-7d87-4e16-a321-1175d0c9f749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:32.587948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8783a1-7d87-4e16-a321-1175d0c9f749 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:32.597874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 756/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:36.334349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73a638-9d0e-4f23-930f-aa497048e426 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:36.334898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73a638-9d0e-4f23-930f-aa497048e426 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:36.334898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73a638-9d0e-4f23-930f-aa497048e426 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:36.336158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:36.341356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8b564-fbf9-4d6d-93a0-90d6c52a0992 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:36.341767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8b564-fbf9-4d6d-93a0-90d6c52a0992 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:36.341912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8b564-fbf9-4d6d-93a0-90d6c52a0992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:36.362164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4cd8b564-fbf9-4d6d-93a0-90d6c52a0992 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:36.363376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4cd8b564-fbf9-4d6d-93a0-90d6c52a0992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:36.371311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:36.680880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73a638-9d0e-4f23-930f-aa497048e426 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:36.682620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2b73a638-9d0e-4f23-930f-aa497048e426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:36.758854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73a638-9d0e-4f23-930f-aa497048e426 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:36.763126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b1bd7-4645-42f0-8c1e-86f65879f4c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:36.763729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b1bd7-4645-42f0-8c1e-86f65879f4c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:36.764012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b1bd7-4645-42f0-8c1e-86f65879f4c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:36.764285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b1bd7-4645-42f0-8c1e-86f65879f4c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:36.773608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b1bd7-4645-42f0-8c1e-86f65879f4c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:36.775037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b1bd7-4645-42f0-8c1e-86f65879f4c6 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:36.785750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 779/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:38.901650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e615a-ef46-46e2-bb73-c99a47ecc694 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:38.902134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e615a-ef46-46e2-bb73-c99a47ecc694 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:38.902248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e615a-ef46-46e2-bb73-c99a47ecc694 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:38.903590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 14:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:55:38.908780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ac3ea-0701-457e-abf4-60c787cc5762 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:38.909224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ac3ea-0701-457e-abf4-60c787cc5762 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:38.909355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ac3ea-0701-457e-abf4-60c787cc5762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:38.928592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-203ac3ea-0701-457e-abf4-60c787cc5762 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:55:38.929808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-203ac3ea-0701-457e-abf4-60c787cc5762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:38.939295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 14:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:55:39.254028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e615a-ef46-46e2-bb73-c99a47ecc694 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:55:39.255732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3a0e615a-ef46-46e2-bb73-c99a47ecc694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:55:39.270025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e615a-ef46-46e2-bb73-c99a47ecc694 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:55:39.273848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07bd79-ad11-4805-9212-b8d479107fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:55:39.274852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07bd79-ad11-4805-9212-b8d479107fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:55:39.275366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07bd79-ad11-4805-9212-b8d479107fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:55:39.275899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07bd79-ad11-4805-9212-b8d479107fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:55:39.287228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07bd79-ad11-4805-9212-b8d479107fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:55:39.288668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07bd79-ad11-4805-9212-b8d479107fa5 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:55:39.298386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 759/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:55:48.923799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=63 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:55:49.293952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=37 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:02.583840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993ae3-0e30-4456-a7eb-5ecb049d4f26 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.584545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993ae3-0e30-4456-a7eb-5ecb049d4f26 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.584545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993ae3-0e30-4456-a7eb-5ecb049d4f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.609878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-56993ae3-0e30-4456-a7eb-5ecb049d4f26 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:02.611126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-56993ae3-0e30-4456-a7eb-5ecb049d4f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:02.620864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 781/1540] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:02.630437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b4441-027c-4f46-85af-85a319f3f684 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.631407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b4441-027c-4f46-85af-85a319f3f684 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.631407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b4441-027c-4f46-85af-85a319f3f684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.643014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582de77-115a-4b81-bdf2-2b05f39564c9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.643441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582de77-115a-4b81-bdf2-2b05f39564c9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.643866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582de77-115a-4b81-bdf2-2b05f39564c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.666078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8b4b4441-027c-4f46-85af-85a319f3f684 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:02.667332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8b4b4441-027c-4f46-85af-85a319f3f684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:02.676937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 760/1541] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:02.680805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0582de77-115a-4b81-bdf2-2b05f39564c9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:02.681681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b4441-027c-4f46-85af-85a319f3f684 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:02.682085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0582de77-115a-4b81-bdf2-2b05f39564c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:02.694977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b4441-027c-4f46-85af-85a319f3f684 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:02.700446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:02.701067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.701367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.701638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.702289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:02.702590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:02.710085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:57:02.710085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Sep 09 14:57:02.710085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: Sep 09 14:57:02.711007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:02.711588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 782/1542] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 09 14:57:02.720794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca85b-6bc2-4253-a512-73bee8bf4a3a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.721111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca85b-6bc2-4253-a512-73bee8bf4a3a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.721369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca85b-6bc2-4253-a512-73bee8bf4a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.725568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 761/1543] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:57:02.734099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:02.745262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2880a25-736c-4359-81fd-74428cd8c4fe admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:02.755009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:02.755972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.756252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.756524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.757158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:02.758508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bddca85b-6bc2-4253-a512-73bee8bf4a3a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:02.759100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:02.759733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bddca85b-6bc2-4253-a512-73bee8bf4a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:02.787416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 783/1544] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:57:02.788710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52a824-f8dc-44e7-bea2-e8675951d3ac None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.788993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52a824-f8dc-44e7-bea2-e8675951d3ac None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.789288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52a824-f8dc-44e7-bea2-e8675951d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.830325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 762/1545] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:02 2022] POST /identity/v3/projects => generated 357 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:57:02.831286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:02.842961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-da52a824-f8dc-44e7-bea2-e8675951d3ac None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:02.850903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-da52a824-f8dc-44e7-bea2-e8675951d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:02.947811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 784/1546] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:02.949472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52a824-f8dc-44e7-bea2-e8675951d3ac None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:02.962849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52a824-f8dc-44e7-bea2-e8675951d3ac None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:02.969139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2c640-7cab-45e3-ae34-6a175c78a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:02.969985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2c640-7cab-45e3-ae34-6a175c78a0db admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.970600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2c640-7cab-45e3-ae34-6a175c78a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.971034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2c640-7cab-45e3-ae34-6a175c78a0db admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.972371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2c640-7cab-45e3-ae34-6a175c78a0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:02.975080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2c640-7cab-45e3-ae34-6a175c78a0db admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:02.975556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d23c7-ff69-4064-9e49-99a60b462b74 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:02.981309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c18025-5e44-4d44-9d71-ff65aa50bf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:02.981929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c18025-5e44-4d44-9d71-ff65aa50bf08 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:02.982208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c18025-5e44-4d44-9d71-ff65aa50bf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:02.982862 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c18025-5e44-4d44-9d71-ff65aa50bf08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:02.983569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c18025-5e44-4d44-9d71-ff65aa50bf08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:02.983878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c18025-5e44-4d44-9d71-ff65aa50bf08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:02.985497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c18025-5e44-4d44-9d71-ff65aa50bf08 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.002575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 763/1547] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:03.004540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082db6e-c2f3-41a0-b937-76f2f287d8e4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.005136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082db6e-c2f3-41a0-b937-76f2f287d8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.005669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082db6e-c2f3-41a0-b937-76f2f287d8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.010562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 785/1548] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:02 2022] POST /identity/v3/users => generated 279 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.012395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf2792-948c-4b59-b263-37eace7c7755 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.012680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf2792-948c-4b59-b263-37eace7c7755 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.012957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf2792-948c-4b59-b263-37eace7c7755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.031918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5082db6e-c2f3-41a0-b937-76f2f287d8e4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.033135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5082db6e-c2f3-41a0-b937-76f2f287d8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.035678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-12bf2792-948c-4b59-b263-37eace7c7755 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.036878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-12bf2792-948c-4b59-b263-37eace7c7755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.041298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 764/1549] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.049228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 786/1550] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.049228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26696d18-1340-48f7-b6e3-52c0207c1eff None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.049228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26696d18-1340-48f7-b6e3-52c0207c1eff None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.049228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26696d18-1340-48f7-b6e3-52c0207c1eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.049674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6ce31-47bf-4e83-858f-cb1b56eedf3f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.049674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6ce31-47bf-4e83-858f-cb1b56eedf3f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.049674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6ce31-47bf-4e83-858f-cb1b56eedf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.065622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-26696d18-1340-48f7-b6e3-52c0207c1eff None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.066808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-26696d18-1340-48f7-b6e3-52c0207c1eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.069341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2bd6ce31-47bf-4e83-858f-cb1b56eedf3f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.070512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2bd6ce31-47bf-4e83-858f-cb1b56eedf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.075162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 787/1551] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.076307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eb19f-2929-4156-9fe4-f43d499cd5df None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.076588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eb19f-2929-4156-9fe4-f43d499cd5df None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.076853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eb19f-2929-4156-9fe4-f43d499cd5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.081595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 765/1552] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.082738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02913a0-6464-4cc4-9032-d28b751f6ee3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.083027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02913a0-6464-4cc4-9032-d28b751f6ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.083299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02913a0-6464-4cc4-9032-d28b751f6ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.101378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ef7eb19f-2929-4156-9fe4-f43d499cd5df None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.102499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ef7eb19f-2929-4156-9fe4-f43d499cd5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.103373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e02913a0-6464-4cc4-9032-d28b751f6ee3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.104549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e02913a0-6464-4cc4-9032-d28b751f6ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.110498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 788/1553] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.111246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eb19f-2929-4156-9fe4-f43d499cd5df None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.114735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 766/1554] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.120954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eb19f-2929-4156-9fe4-f43d499cd5df None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.121642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02913a0-6464-4cc4-9032-d28b751f6ee3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.126061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ba702-5c40-487a-9c72-2ce3da6e3495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.126678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ba702-5c40-487a-9c72-2ce3da6e3495 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.126954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ba702-5c40-487a-9c72-2ce3da6e3495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.127220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ba702-5c40-487a-9c72-2ce3da6e3495 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.127868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ba702-5c40-487a-9c72-2ce3da6e3495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.128884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-c68ba702-5c40-487a-9c72-2ce3da6e3495 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:57:03.128884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:57:03.128884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:57:03.130110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ba702-5c40-487a-9c72-2ce3da6e3495 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.131297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02913a0-6464-4cc4-9032-d28b751f6ee3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.136275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8ef8e-950e-43bb-8aef-466da52e9f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.137394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8ef8e-950e-43bb-8aef-466da52e9f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.137659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8ef8e-950e-43bb-8aef-466da52e9f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.137925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8ef8e-950e-43bb-8aef-466da52e9f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.138552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8ef8e-950e-43bb-8aef-466da52e9f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.140760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8ef8e-950e-43bb-8aef-466da52e9f4c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.151613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 767/1555] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:03.153413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779195-5d64-4202-b1fc-37be832ece51 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.153687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779195-5d64-4202-b1fc-37be832ece51 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.153959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779195-5d64-4202-b1fc-37be832ece51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.162129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 789/1556] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.162129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe716-f762-4e37-b245-83b0486d86d9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.162129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe716-f762-4e37-b245-83b0486d86d9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.162129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe716-f762-4e37-b245-83b0486d86d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.175831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9e779195-5d64-4202-b1fc-37be832ece51 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.177012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9e779195-5d64-4202-b1fc-37be832ece51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.180124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-212fe716-f762-4e37-b245-83b0486d86d9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.181335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-212fe716-f762-4e37-b245-83b0486d86d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.240907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 768/1557] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.241928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 790/1558] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.242062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e779195-5d64-4202-b1fc-37be832ece51 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.243031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c487d52-c707-4b77-9aae-c28c18274077 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.243318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c487d52-c707-4b77-9aae-c28c18274077 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.243596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c487d52-c707-4b77-9aae-c28c18274077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.257080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e779195-5d64-4202-b1fc-37be832ece51 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.262401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be9b1a-6081-4140-b9b6-a942e6f671d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.263003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be9b1a-6081-4140-b9b6-a942e6f671d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.263281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be9b1a-6081-4140-b9b6-a942e6f671d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.263383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7c487d52-c707-4b77-9aae-c28c18274077 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.263554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be9b1a-6081-4140-b9b6-a942e6f671d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.264195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be9b1a-6081-4140-b9b6-a942e6f671d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:03.264499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be9b1a-6081-4140-b9b6-a942e6f671d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.264613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7c487d52-c707-4b77-9aae-c28c18274077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.266130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be9b1a-6081-4140-b9b6-a942e6f671d2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.272708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 791/1559] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.276014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304513-f174-477e-8791-bea8a50687f4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.276014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304513-f174-477e-8791-bea8a50687f4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.276269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304513-f174-477e-8791-bea8a50687f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.278626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:03.285496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 769/1560] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:03.285496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb85fba-664f-4410-9934-5b5ed87e51f4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.285496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb85fba-664f-4410-9934-5b5ed87e51f4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.287230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb85fba-664f-4410-9934-5b5ed87e51f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.298458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-59304513-f174-477e-8791-bea8a50687f4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.299666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-59304513-f174-477e-8791-bea8a50687f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.307655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 792/1561] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.309337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-abb85fba-664f-4410-9934-5b5ed87e51f4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.310165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59304513-f174-477e-8791-bea8a50687f4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.310630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-abb85fba-664f-4410-9934-5b5ed87e51f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.319135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 770/1562] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:03.319672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59304513-f174-477e-8791-bea8a50687f4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.319988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb85fba-664f-4410-9934-5b5ed87e51f4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.325003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.326000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.326489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.326963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.328343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.330827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.332305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb85fba-664f-4410-9934-5b5ed87e51f4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.333843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:03.338136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.338626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.338895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.339160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.340647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.341751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:57:03.341751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Sep 09 14:57:03.341751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: Sep 09 14:57:03.342976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.347405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 793/1563] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:03.347405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.357084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f97bd5-c9dd-4b32-9a76-a552ccc5d642 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.363641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.364387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.364882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.365414 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.366162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:03.366304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 771/1564] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.366529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.367049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.368163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.377045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab2f0e-ebe4-4383-8756-bf1a2b1b6e19 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.381056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 794/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:03.381901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.382061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.382699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.382985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.383320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.384022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:03.384387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.386061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.393509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36bbd1-a760-404d-9d09-085eb1cf6982 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.398176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.398786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.399070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.399342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.400228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:03.400528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.402150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.402942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 772/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:03.403859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.413719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb0c45-f4a8-4b5d-97b6-3b52f205dda8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.414356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 795/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:03.415167 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.418740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.419522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.419966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.420412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.421465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.423985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.424954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9d9c6-16bc-475a-8e5c-e62c8fc56ab4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.429696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.430303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.430584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.430858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.431487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:03.431790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.433522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.446373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 796/1568] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:03.454163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.457005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed488db1-52ce-484d-bc53-d01ca9ec78e9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.458933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 773/1569] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.459776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.512537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.513150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.513462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.513737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.514376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:03.514669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.516280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.526184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1473a3-bf5d-436b-8a8c-1dc33ec1e14e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.529133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 797/1570] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:03.530283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.531915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.532749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.533339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.533812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] PATH_INFO: `/v3/projects/fb1ef290da434a6e949d4c18d98c866e/users/fa66856f48284deca2b7f7b0e729c720/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.540729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008610a-cd0d-4ebc-8c5c-70a2040b1ea7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.546147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.546742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.547020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.547401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.547925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.549364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1ef290da434a6e949d4c18d98c866e, user_id=fa66856f48284deca2b7f7b0e729c720, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.549560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.552003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.562311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 798/1571] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:03.563465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.566857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 774/1572] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:03 2022] PUT /identity/v3/projects/fb1ef290da434a6e949d4c18d98c866e/users/fa66856f48284deca2b7f7b0e729c720/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:03.577350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.577462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c1465-4d85-4d88-a8bd-d970d5a97f5b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.630929 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.631602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec5b7a-e119-4ac2-99fa-629a355bc51d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.631971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.631971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.632156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.632693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.634665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.636665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.637954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.638237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.638511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.639166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.641053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.664559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 799/1573] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.664559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.671724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 775/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.671724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.679001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:03.697508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:03.730563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4904a-328f-42ac-9c54-566f074f8900 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.735220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.735819 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.736097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.736367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.736985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7ba8-5298-4d81-803d-4c82fd1fbce5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.737109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.738922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.741748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.742350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.742639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.742913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.743534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.745457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.767079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 800/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.768028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.773916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 776/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.775188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.831003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2abab-aaad-439f-b0f7-ae1c69bcdd08 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.840156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad52ad-3c22-4754-8682-21b9566a52b6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.844900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.845536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.845832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.846092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.846717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.848626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.876487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 777/1577] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.878374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.888102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7fb73-0eb2-478f-a6fd-e81f9d7bd96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.888651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7fb73-0eb2-478f-a6fd-e81f9d7bd96d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.888931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7fb73-0eb2-478f-a6fd-e81f9d7bd96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.889223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7fb73-0eb2-478f-a6fd-e81f9d7bd96d admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.889884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7fb73-0eb2-478f-a6fd-e81f9d7bd96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.892084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7fb73-0eb2-478f-a6fd-e81f9d7bd96d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.913582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 801/1578] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:03.914850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323933d-5f5a-40e1-9844-8b07d9a5c9f3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.915137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323933d-5f5a-40e1-9844-8b07d9a5c9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.915395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323933d-5f5a-40e1-9844-8b07d9a5c9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.941262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997ff01-9d11-4755-8eeb-3e31412925d0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.943000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1323933d-5f5a-40e1-9844-8b07d9a5c9f3 None None] MFA Rules not processed for user `fa66856f48284deca2b7f7b0e729c720`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:03.944202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1323933d-5f5a-40e1-9844-8b07d9a5c9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:03.946243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.946851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.947129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.947432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.948072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.949704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.959575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 778/1579] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:03.960334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:03.969643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ad644-2ede-4eab-9166-653a783b1a90 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:03.974357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:03.974926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:03.975205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:03.975470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:03.976075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:03.977963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:03.983801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:03.995125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:04.003378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 779/1580] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.004780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.019916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 802/1581] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:04.019916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323933d-5f5a-40e1-9844-8b07d9a5c9f3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.020187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0723a-81fa-4123-8d21-c2adb6d11f31 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.024151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.025122 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.025436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.025774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.026421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.028298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.029487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323933d-5f5a-40e1-9844-8b07d9a5c9f3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.034172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.034792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.035064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.035329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.035978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.037942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.050376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 780/1582] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.051335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.063055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3f7e9-b2aa-4e73-9b7c-83f598697875 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.065005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 803/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/users => generated 283 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.065915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.068278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.068882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.069164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.069477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.070106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.072823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.075330 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7df52-2631-4c61-80ea-b13e235c0873 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.079952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.080524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.080808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.081079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.081728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.083634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.096047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 781/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.097080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.105571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 804/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/users => generated 290 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.107301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.107985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398e300-438c-4fb7-a78c-41121cdccf75 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.112706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.113364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.113717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.113925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.114607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.116280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.117834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55293630-5059-4a0a-a766-61c67526da6c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.122925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.123561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.123840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.124106 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] PATH_INFO: `/v3/projects/2ef9057fcdbb4cef93ccf051e4ef2231/users/9ef5ed0139254c1981166c806f70d036/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.130233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 782/1586] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.130233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.137774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac4e7-5793-4383-bab8-aafb976ebc76 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.142441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef9057fcdbb4cef93ccf051e4ef2231, user_id=9ef5ed0139254c1981166c806f70d036, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.142565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.142964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.143243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.143412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:57:04.143412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:57:04.143412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:57:04.143672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.144132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.145372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.145754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.156881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 783/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.160019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.162000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 805/1588] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:04 2022] PUT /identity/v3/projects/2ef9057fcdbb4cef93ccf051e4ef2231/users/9ef5ed0139254c1981166c806f70d036/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:04.162802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.167992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852abf25-a090-4264-b0ec-1ee194b44028 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.224603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.225542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.225831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.226084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.226475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d617b2-c9e4-4512-8474-4760464cef5b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.226706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.228598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.232516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.233112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.233420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.233691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.234319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.235899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.245668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 806/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.246495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.256916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76858147-dbd4-4109-bd73-05600851e5e3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.263789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 784/1590] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.263789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.264662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.265301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.265591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.265866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.266501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.268094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.279118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 807/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.280535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.290901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba15fe8-b02d-4544-8874-7ecb521a994b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.325830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022c098-035e-4102-b305-71c9e711873b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.332046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.332643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.332920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.333191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.334617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.336216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.349365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.350253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.350673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.351085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.351449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 785/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.352067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.352275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.354823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.363182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:04.364478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b739b5c-23a4-4651-9a37-0150cb5d7ae5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.369400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d0a74-e86c-4b44-bf5b-fd28cf4ec188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.369994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0a74-e86c-4b44-bf5b-fd28cf4ec188 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.370274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0a74-e86c-4b44-bf5b-fd28cf4ec188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.370548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0a74-e86c-4b44-bf5b-fd28cf4ec188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.371256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d0a74-e86c-4b44-bf5b-fd28cf4ec188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.371385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 808/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.372834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.372978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d0a74-e86c-4b44-bf5b-fd28cf4ec188 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.383355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 786/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.384701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa9a4d-debc-402d-acc8-9fa4223fcea5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.384991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa9a4d-debc-402d-acc8-9fa4223fcea5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.385288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa9a4d-debc-402d-acc8-9fa4223fcea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.386034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34bf4-88f0-4c24-8dbc-3589c8d345ce admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.390801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.391387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.391669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.391936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.392617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.394210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.405350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 809/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.406145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.411189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:04.416517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a9131-c2a8-40b9-9fda-58d0e6d0479e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.421874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e6aa9a4d-debc-402d-acc8-9fa4223fcea5 None None] MFA Rules not processed for user `9ef5ed0139254c1981166c806f70d036`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:04.422273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.422853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.423128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.423398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.424026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.425177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e6aa9a4d-debc-402d-acc8-9fa4223fcea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:04.425719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.435729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 810/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.436484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.448234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c4ba9-d37b-4b0a-99ca-78a0cc588d4b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.453005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.453609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.453888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.454154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.454793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.456682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.482070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 811/1597] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/users => generated 284 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.482070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.489134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c331d-3eb7-4529-94d5-021bf72a0187 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.490830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 787/1598] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:04.491592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa9a4d-debc-402d-acc8-9fa4223fcea5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.493810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.494394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.494670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.494992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.495618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.497303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.502095 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa9a4d-debc-402d-acc8-9fa4223fcea5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.506848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.507443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.507545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 812/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.507758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.508017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.508209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.508653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.510323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.520575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef18323-e388-4b6f-99da-8155f53de0e3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.524105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 788/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.524105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.530169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.530826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.531105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.531374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.532004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.532766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13efbb-6dfb-49d7-922d-bdc8126cc874 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.533678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.537846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.538483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.538772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.539044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] PATH_INFO: `/v3/projects/b8051535f56a4e17a0fa1eeab7c08ce7/users/cd6c40dd492e40599324ef05011895f3/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.543722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 813/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.544492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.553515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8051535f56a4e17a0fa1eeab7c08ce7, user_id=cd6c40dd492e40599324ef05011895f3, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.556171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.556904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921dfd2-8fa4-4cf7-865c-74f2f94b94a1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.561816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.562736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.563016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.563911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] PATH_INFO: `/v3/projects/571292c3d1e94192b8a841cc4df40549/users/1089c2be3fa14a1890f901d27576f4f0/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.570141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 789/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:04 2022] PUT /identity/v3/projects/b8051535f56a4e17a0fa1eeab7c08ce7/users/cd6c40dd492e40599324ef05011895f3/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:04.570941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.580239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=571292c3d1e94192b8a841cc4df40549, user_id=1089c2be3fa14a1890f901d27576f4f0, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.582928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.597738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 814/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:04 2022] PUT /identity/v3/projects/571292c3d1e94192b8a841cc4df40549/users/1089c2be3fa14a1890f901d27576f4f0/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:04.598541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.644433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7016f-b355-4019-a6f6-44a2f622e3fc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.659325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a49ec-e770-463c-b241-fffe4d840c70 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.664227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.664820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.665096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.665396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.666021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.667598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.677225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 815/1604] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:04.677793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.686977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733008ed-b434-4c1c-a6c6-65f9384ba8b6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.691939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.692514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.692812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.693056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.693701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.695564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.700557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.701258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.701542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.701816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] PATH_INFO: `/v3/projects/c83910e3ac2f4c8f9f127ab767bffd1a/users/3c0c848f7858486c91773b34047d8aad/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.719167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83910e3ac2f4c8f9f127ab767bffd1a, user_id=3c0c848f7858486c91773b34047d8aad, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.726909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 816/1605] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:04.727117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.727220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.728238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:04.741046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 790/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:04 2022] PUT /identity/v3/projects/c83910e3ac2f4c8f9f127ab767bffd1a/users/3c0c848f7858486c91773b34047d8aad/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:04.742080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.774180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:04.790443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ca718-6a12-4ca8-af8e-e2908107eb61 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.804124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a8fe7-1fbe-44c7-892c-6359b4a2e628 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.808726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.809456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.809753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.810022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] PATH_INFO: `/v3/projects/063bdf7448ee45939287687b0e65dd97/users/770105b06e084179a819a54d28b70cdf/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.824238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063bdf7448ee45939287687b0e65dd97, user_id=770105b06e084179a819a54d28b70cdf, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.827448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.840871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 791/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:04 2022] PUT /identity/v3/projects/063bdf7448ee45939287687b0e65dd97/users/770105b06e084179a819a54d28b70cdf/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:04.841670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:04.847193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692a3a7-cc44-45ef-893f-7c32bededfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.847987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692a3a7-cc44-45ef-893f-7c32bededfff admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.848270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692a3a7-cc44-45ef-893f-7c32bededfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.848542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692a3a7-cc44-45ef-893f-7c32bededfff admin admin] PATH_INFO: `/v3/projects/dbf5912f0c324ba3bb5312868183c630/users/4c0b5403b8814a68b4615a787f79c32f/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.863126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692a3a7-cc44-45ef-893f-7c32bededfff admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf5912f0c324ba3bb5312868183c630, user_id=4c0b5403b8814a68b4615a787f79c32f, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.865902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692a3a7-cc44-45ef-893f-7c32bededfff admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.879794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 817/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:04 2022] PUT /identity/v3/projects/dbf5912f0c324ba3bb5312868183c630/users/4c0b5403b8814a68b4615a787f79c32f/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:04.881251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ac4e-4d4d-43f5-b8ed-dde9e7dd6abc None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.881703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ac4e-4d4d-43f5-b8ed-dde9e7dd6abc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.881980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ac4e-4d4d-43f5-b8ed-dde9e7dd6abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.906469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d211ac4e-4d4d-43f5-b8ed-dde9e7dd6abc None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:04.907622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d211ac4e-4d4d-43f5-b8ed-dde9e7dd6abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:04.908098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3c0dc-6a7b-4594-a433-46d2468123c3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:04.966765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:04.967339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 818/1609] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:04.967469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.967694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.967965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] PATH_INFO: `/v3/projects/3c2219444c1b4ad0a395880494ddbbe3/users/1e123765a8724a89863d74221001dcd1/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.968655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c2389-3b72-4598-b832-dd6a6f8dd001 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:04.968944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c2389-3b72-4598-b832-dd6a6f8dd001 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:04.969243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c2389-3b72-4598-b832-dd6a6f8dd001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:04.985852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2219444c1b4ad0a395880494ddbbe3, user_id=1e123765a8724a89863d74221001dcd1, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:04.988494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:04.997601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-de2c2389-3b72-4598-b832-dd6a6f8dd001 None None] MFA Rules not processed for user `1089c2be3fa14a1890f901d27576f4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:04.998813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-de2c2389-3b72-4598-b832-dd6a6f8dd001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:05.002168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 792/1610] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:04 2022] PUT /identity/v3/projects/3c2219444c1b4ad0a395880494ddbbe3/users/1e123765a8724a89863d74221001dcd1/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:05.005897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.015970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:05.066651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 819/1611] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:05.067531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c2389-3b72-4598-b832-dd6a6f8dd001 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.072243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd9fd-226d-4110-b650-c73c9fc5defa admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.077548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dcc7b-cf24-4644-b390-d48024deddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.078057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c2389-3b72-4598-b832-dd6a6f8dd001 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.078310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dcc7b-cf24-4644-b390-d48024deddee admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.078595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dcc7b-cf24-4644-b390-d48024deddee admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.078867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dcc7b-cf24-4644-b390-d48024deddee admin admin] PATH_INFO: `/v3/projects/5eff1d3c3d4e43cc87b3390db10cdbb4/users/755e26e7fcb944fc820e7a74cb3e4620/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.082738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbdc57-e916-4abe-a1b3-b96445c9ef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.083315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbdc57-e916-4abe-a1b3-b96445c9ef6a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.083594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbdc57-e916-4abe-a1b3-b96445c9ef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.083865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbdc57-e916-4abe-a1b3-b96445c9ef6a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.084657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbdc57-e916-4abe-a1b3-b96445c9ef6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.086620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbdc57-e916-4abe-a1b3-b96445c9ef6a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.094264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dcc7b-cf24-4644-b390-d48024deddee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eff1d3c3d4e43cc87b3390db10cdbb4, user_id=755e26e7fcb944fc820e7a74cb3e4620, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.096124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:05.096910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dcc7b-cf24-4644-b390-d48024deddee admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.111488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 820/1612] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:05.111782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 793/1613] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:05 2022] PUT /identity/v3/projects/5eff1d3c3d4e43cc87b3390db10cdbb4/users/755e26e7fcb944fc820e7a74cb3e4620/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:05.113009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48449304-b2fd-4314-accd-f9f49a63a93d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.113320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48449304-b2fd-4314-accd-f9f49a63a93d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.113595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48449304-b2fd-4314-accd-f9f49a63a93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.113726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac8e2a-1c0f-4705-9183-7c83f1f99446 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.114153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac8e2a-1c0f-4705-9183-7c83f1f99446 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.114577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac8e2a-1c0f-4705-9183-7c83f1f99446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.132781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-48449304-b2fd-4314-accd-f9f49a63a93d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:05.134006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-faac8e2a-1c0f-4705-9183-7c83f1f99446 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:05.134166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-48449304-b2fd-4314-accd-f9f49a63a93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:05.135168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-faac8e2a-1c0f-4705-9183-7c83f1f99446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:05.193039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 794/1614] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:05.194235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec0aa-7c7c-466f-bdbb-55c4e9a4a361 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.194517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec0aa-7c7c-466f-bdbb-55c4e9a4a361 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.194616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 821/1615] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:05.194792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec0aa-7c7c-466f-bdbb-55c4e9a4a361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.195478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac8e2a-1c0f-4705-9183-7c83f1f99446 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.224666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-105ec0aa-7c7c-466f-bdbb-55c4e9a4a361 None None] MFA Rules not processed for user `4c0b5403b8814a68b4615a787f79c32f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:05.225931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-105ec0aa-7c7c-466f-bdbb-55c4e9a4a361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:05.251766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac8e2a-1c0f-4705-9183-7c83f1f99446 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.255594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7617a5-1856-452f-9862-6ee41f12555d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.256189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7617a5-1856-452f-9862-6ee41f12555d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.256461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7617a5-1856-452f-9862-6ee41f12555d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.256726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7617a5-1856-452f-9862-6ee41f12555d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.266149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7617a5-1856-452f-9862-6ee41f12555d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.267569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7617a5-1856-452f-9862-6ee41f12555d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.277071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 822/1616] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:05.278831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a4747-bb9d-4334-a8a0-9409bdf5b970 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.279112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a4747-bb9d-4334-a8a0-9409bdf5b970 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.279401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a4747-bb9d-4334-a8a0-9409bdf5b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.295774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 795/1617] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:05.296729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ec0aa-7c7c-466f-bdbb-55c4e9a4a361 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.304338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3d5a4747-bb9d-4334-a8a0-9409bdf5b970 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:05.305739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3d5a4747-bb9d-4334-a8a0-9409bdf5b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:05.309608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ec0aa-7c7c-466f-bdbb-55c4e9a4a361 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.314213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.314907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.315331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.315647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.319069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.319361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 823/1618] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:05.319361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a4747-bb9d-4334-a8a0-9409bdf5b970 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.319558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.330844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a4747-bb9d-4334-a8a0-9409bdf5b970 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.336212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.336894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.337296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.337584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.338205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.339898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.348558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:05.353266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 824/1619] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:05.357447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.362797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 796/1620] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:05.362797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.368609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6ab54-128b-487c-b9eb-c7b09dd1655c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.426874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80bc6b-fec2-468a-b052-5ca065122e82 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.430926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.431943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.432299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.432670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.438291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.439291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.439957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.440367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.441378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.444590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.444960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.446169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.458073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 797/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:05.461153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 825/1622] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:05.461298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.461398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.461502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:05.479677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:05.481830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290501a6-c115-4097-940c-e30a3010e7f0 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.486912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.487528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.487833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.488897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.502835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.506771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.525553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 798/1623] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:05.527224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.532939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0566a-459f-4422-9ece-99beb1d21975 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.537477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.538454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.538754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05768a-8586-4984-8e0a-40969ff9d5fa service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.539108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.539523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.543832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.544565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.544942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.545398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.565180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.566664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.578950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 799/1624] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:05.580236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.593026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df476c-8d2d-4a13-8d67-83fd2e8b2930 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.601503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.602452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.602988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.603321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] PATH_INFO: `/v3/projects/5eff1d3c3d4e43cc87b3390db10cdbb4/users/755e26e7fcb944fc820e7a74cb3e4620/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.606865 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eff1d3c3d4e43cc87b3390db10cdbb4, user_id=755e26e7fcb944fc820e7a74cb3e4620, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.609954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.624767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 800/1625] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:05 2022] PUT /identity/v3/projects/5eff1d3c3d4e43cc87b3390db10cdbb4/users/755e26e7fcb944fc820e7a74cb3e4620/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:05.625641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.628111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.635044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.645572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 826/1626] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:05.646717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.666516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4fe7-4955-46e3-9edc-4ff0d1fe3677 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.740490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b410f5-cf89-4622-b504-ea1be9d35d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.741349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b410f5-cf89-4622-b504-ea1be9d35d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.741632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b410f5-cf89-4622-b504-ea1be9d35d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.741904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b410f5-cf89-4622-b504-ea1be9d35d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.742554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b410f5-cf89-4622-b504-ea1be9d35d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.744185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b410f5-cf89-4622-b504-ea1be9d35d13 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.756213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd3981-e9cb-4341-bb2e-d671a9e530c3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.757655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 827/1627] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:05.758979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d816-46bc-4c63-8bbe-f11aa1995bd1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.759269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d816-46bc-4c63-8bbe-f11aa1995bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.759546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d816-46bc-4c63-8bbe-f11aa1995bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.773625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.773625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.773943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.774196 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.775175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.780762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.801702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fd99d816-46bc-4c63-8bbe-f11aa1995bd1 None None] MFA Rules not processed for user `755e26e7fcb944fc820e7a74cb3e4620`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:05.803236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fd99d816-46bc-4c63-8bbe-f11aa1995bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:05.825229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 801/1628] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:05.826034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.839029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727090-56fc-48d4-bec5-418ff9803e4f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:05.845386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:05.846069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.846459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.846714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] PATH_INFO: `/v3/projects/31669bdc8f4a4b6c9bfc3b4d018333a2/users/70b106efba964581a87932573ea17b26/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.866496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31669bdc8f4a4b6c9bfc3b4d018333a2, user_id=70b106efba964581a87932573ea17b26, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:05.870222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:05.881569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:05.897099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 802/1629] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:05 2022] PUT /identity/v3/projects/31669bdc8f4a4b6c9bfc3b4d018333a2/users/70b106efba964581a87932573ea17b26/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:05.897983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:05.924007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 828/1630] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:05.926039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2d2c1-21fa-46df-b329-1606a8e422f2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:05.926366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2d2c1-21fa-46df-b329-1606a8e422f2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:05.927140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2d2c1-21fa-46df-b329-1606a8e422f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:05.978020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:05.983421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d9a2d2c1-21fa-46df-b329-1606a8e422f2 None None] MFA Rules not processed for user `70b106efba964581a87932573ea17b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:05.987882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d9a2d2c1-21fa-46df-b329-1606a8e422f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:06.064583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6831d-e293-4d18-a87c-5e2a9968fe1f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.079555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.080187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.080461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.080730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.081417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.083012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.104894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 803/1631] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:06.106341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.117328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 829/1632] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:06.126948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2d2c1-21fa-46df-b329-1606a8e422f2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.147648 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2d2c1-21fa-46df-b329-1606a8e422f2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.154801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.156013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.156309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.156596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.157863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.160466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.180228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 830/1633] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:06.186380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.203538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b743a9-d02d-46fe-bb50-edb2e225269b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.209272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.209982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.210271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.210547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.211239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.213658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.223943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e622b3-5c9a-4f87-a1dc-fed33181b013 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.233851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.236530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.237767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.238227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.266320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 831/1634] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:06 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:06.268295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.425709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.427465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.438472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3464e2-1c31-4c91-9a82-56e4dd979fdb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.448189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 804/1635] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:06.449123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.466035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31d366-7aa1-40dc-bcdc-4c500e359233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.467065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31d366-7aa1-40dc-bcdc-4c500e359233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.467508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31d366-7aa1-40dc-bcdc-4c500e359233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.467930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31d366-7aa1-40dc-bcdc-4c500e359233 admin admin] PATH_INFO: `/v3/projects/c1fe70fb7c2a4e4ca6be0e60838a8837/users/68a16b43aad7402ebcb7382f46e1cc52/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.472676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5630be-47aa-4205-807f-f2f1dbbd0e4e service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.485704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.486310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.486663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.487004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.487717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.490336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.518060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31d366-7aa1-40dc-bcdc-4c500e359233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fe70fb7c2a4e4ca6be0e60838a8837, user_id=68a16b43aad7402ebcb7382f46e1cc52, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.520625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31d366-7aa1-40dc-bcdc-4c500e359233 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.541639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 805/1636] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:06 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:06.542859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.561291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 832/1637] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:06 2022] PUT /identity/v3/projects/c1fe70fb7c2a4e4ca6be0e60838a8837/users/68a16b43aad7402ebcb7382f46e1cc52/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:06.565496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ba2e-6846-4d00-9e25-1d31f1fc0dd3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.574011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59cce5-960a-46a9-a8a6-0d0cafc03aa2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.574864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59cce5-960a-46a9-a8a6-0d0cafc03aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.575318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59cce5-960a-46a9-a8a6-0d0cafc03aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.634442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8b59cce5-960a-46a9-a8a6-0d0cafc03aa2 None None] MFA Rules not processed for user `68a16b43aad7402ebcb7382f46e1cc52`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:06.637255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8b59cce5-960a-46a9-a8a6-0d0cafc03aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:06.644302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:06.653643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.654684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.655152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.655666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.656720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.659011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.676313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 806/1638] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:06.677471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:06.679504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.702565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e53386-b8bc-4241-ae26-3092922c5233 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.711335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.711808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.712219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.712565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] PATH_INFO: `/v3/projects/405c1623ad1a42b38fbab864d1feda77/users/84d213651aec43928ac23ed2726ec3e0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.743053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405c1623ad1a42b38fbab864d1feda77, user_id=84d213651aec43928ac23ed2726ec3e0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.748535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:06.780558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 833/1639] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:57:06.781531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59cce5-960a-46a9-a8a6-0d0cafc03aa2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.800421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 807/1640] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:06 2022] PUT /identity/v3/projects/405c1623ad1a42b38fbab864d1feda77/users/84d213651aec43928ac23ed2726ec3e0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:06.804694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:06.852943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59cce5-960a-46a9-a8a6-0d0cafc03aa2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.947635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664881f-8b8b-446f-9be2-5ce392dd7b29 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:06.948507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.949132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.949981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.950344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.961734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:06.962355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:06.962732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:06.963009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:06.964092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:06.966154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.017636 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.020372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.031822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 808/1641] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:06 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:07.033061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.047286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 834/1642] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:07.049352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.071001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caab675-d945-44a3-9fa2-afae2cbea1ff service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.127486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:07.165914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c19ae5-cb7a-4b75-a946-a38c8ecd39ba admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.175415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.176202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.176539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.176892 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.177644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.179442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.194576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 809/1643] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:07.195515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.212957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.212957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.212957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.213426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.214308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70332-03b6-44cd-8db4-e20caeac0bb2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.215541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.218701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.228513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b1b50-dba6-4687-b45c-6f2e059d8438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.229332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b1b50-dba6-4687-b45c-6f2e059d8438 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.229674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b1b50-dba6-4687-b45c-6f2e059d8438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.230504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b1b50-dba6-4687-b45c-6f2e059d8438 admin admin] PATH_INFO: `/v3/projects/405c1623ad1a42b38fbab864d1feda77/users/84d213651aec43928ac23ed2726ec3e0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.235107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b1b50-dba6-4687-b45c-6f2e059d8438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405c1623ad1a42b38fbab864d1feda77, user_id=84d213651aec43928ac23ed2726ec3e0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.238989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b1b50-dba6-4687-b45c-6f2e059d8438 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.265094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 835/1644] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:07 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:07.278335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 810/1645] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:07 2022] PUT /identity/v3/projects/405c1623ad1a42b38fbab864d1feda77/users/84d213651aec43928ac23ed2726ec3e0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:07.279718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.281473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f348e-6eb6-480e-9fec-ec48eb4689fb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.282602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f348e-6eb6-480e-9fec-ec48eb4689fb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.282947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f348e-6eb6-480e-9fec-ec48eb4689fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.300645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bed89-5ff3-46ef-a74e-50e2a10c16d2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.347473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-594f348e-6eb6-480e-9fec-ec48eb4689fb None None] MFA Rules not processed for user `84d213651aec43928ac23ed2726ec3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:07.349631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-594f348e-6eb6-480e-9fec-ec48eb4689fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:07.410704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:07.436773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.438036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.440261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.440704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.441464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.444884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.469590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 836/1646] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:07.475116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.500661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545db6-e5d3-420b-9dd3-7fa1a5f04f4b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.508879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 811/1647] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:07.515530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f348e-6eb6-480e-9fec-ec48eb4689fb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.518360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.519595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.519974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.520438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] PATH_INFO: `/v3/projects/6722a8ce619f44ec8f63cb969ae8f460/users/d86692123ae8418a8126d54291469049/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.558041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6722a8ce619f44ec8f63cb969ae8f460, user_id=d86692123ae8418a8126d54291469049, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.562491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.595127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 837/1648] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:07 2022] PUT /identity/v3/projects/6722a8ce619f44ec8f63cb969ae8f460/users/d86692123ae8418a8126d54291469049/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:07.599478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.654608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f348e-6eb6-480e-9fec-ec48eb4689fb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.735856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f4ba-2874-43e4-b52e-6f48f3f26d94 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.750360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.750994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.755018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.755358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.756044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.758013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.779484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.780625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.782741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.783270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.793765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 838/1649] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:57:07.795217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:07.816490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc344f4-7f02-4fe0-82f8-e40056dafba0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:07.831417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54d112-7a6f-4a3b-9d57-6be4026bb10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:07.833501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d112-7a6f-4a3b-9d57-6be4026bb10f admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.835517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d112-7a6f-4a3b-9d57-6be4026bb10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.835817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d112-7a6f-4a3b-9d57-6be4026bb10f admin admin] PATH_INFO: `/v3/projects/6722a8ce619f44ec8f63cb969ae8f460/users/d86692123ae8418a8126d54291469049/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.840130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54d112-7a6f-4a3b-9d57-6be4026bb10f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6722a8ce619f44ec8f63cb969ae8f460, user_id=d86692123ae8418a8126d54291469049, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.848239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54d112-7a6f-4a3b-9d57-6be4026bb10f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:07.875871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 839/1650] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:07 2022] PUT /identity/v3/projects/6722a8ce619f44ec8f63cb969ae8f460/users/d86692123ae8418a8126d54291469049/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:07.890341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecce32-8349-4e3d-9901-ec1c47aef9db None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:07.890650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecce32-8349-4e3d-9901-ec1c47aef9db None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:07.890882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecce32-8349-4e3d-9901-ec1c47aef9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:07.945448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:07.946949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ccecce32-8349-4e3d-9901-ec1c47aef9db None None] MFA Rules not processed for user `d86692123ae8418a8126d54291469049`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:07.948388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ccecce32-8349-4e3d-9901-ec1c47aef9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:07.979458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:07.981738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:08.005941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 812/1651] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:57:08.118452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 840/1652] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:57:08.135320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:08.153570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84513b1d-932d-4088-aac9-e9cd873a0a76 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:08.215754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:08.257292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277e58c-b4b3-409d-9962-4ab3bb5f8182 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:08.257774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277e58c-b4b3-409d-9962-4ab3bb5f8182 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:08.258047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277e58c-b4b3-409d-9962-4ab3bb5f8182 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:08.259903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 841/1653] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 14:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:08.276978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:08.280130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:08.280974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:08.282207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a622f-1d1f-4b38-8f39-a2e76893d09f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:08.282344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:08.282441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a622f-1d1f-4b38-8f39-a2e76893d09f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:08.282542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a622f-1d1f-4b38-8f39-a2e76893d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:08.301793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:08.303894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:08.325022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 813/1654] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:08.350578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fa2a622f-1d1f-4b38-8f39-a2e76893d09f None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:08.352258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fa2a622f-1d1f-4b38-8f39-a2e76893d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:08.420960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:08.441748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05353432-fe3b-4205-adb5-5b5ccd11dda0 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:08.455591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:08.456236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:08.456521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:08.456797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:08.555035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 842/1655] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 14:57:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:08.559509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b1c4-49cf-4d07-8ce4-3c3f8bc520ea None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:08.559793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b1c4-49cf-4d07-8ce4-3c3f8bc520ea None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:08.560070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b1c4-49cf-4d07-8ce4-3c3f8bc520ea None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:08.571427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 843/1656] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 14:57:08 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:08.576524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f870c-56d4-471f-ae64-1aa3acfc6157 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:08.576524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f870c-56d4-471f-ae64-1aa3acfc6157 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:08.577044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f870c-56d4-471f-ae64-1aa3acfc6157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:08.620273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:08.622341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:08.639990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-777f870c-56d4-471f-ae64-1aa3acfc6157 None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:08.641248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-777f870c-56d4-471f-ae64-1aa3acfc6157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:08.653500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 814/1657] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:57:08.672575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 844/1658] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:57:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:08.710229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:08.720605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defefdc8-db84-4bdd-bac4-fcc8569bb31c service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:08.726839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6108e-be43-48ee-9fc6-2137f55f5cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:08.726839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6108e-be43-48ee-9fc6-2137f55f5cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:08.727164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6108e-be43-48ee-9fc6-2137f55f5cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:08.727438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6108e-be43-48ee-9fc6-2137f55f5cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:08.743150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6108e-be43-48ee-9fc6-2137f55f5cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:08.745972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6108e-be43-48ee-9fc6-2137f55f5cd7 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:08.773432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 815/1659] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:10.029840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f870c-56d4-471f-ae64-1aa3acfc6157 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:10.046751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f870c-56d4-471f-ae64-1aa3acfc6157 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:10.054484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02bc82-c0fa-4913-9015-2e58b964d306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:10.056459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02bc82-c0fa-4913-9015-2e58b964d306 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:10.058755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02bc82-c0fa-4913-9015-2e58b964d306 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:10.060121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02bc82-c0fa-4913-9015-2e58b964d306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:10.231987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02bc82-c0fa-4913-9015-2e58b964d306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:10.235674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02bc82-c0fa-4913-9015-2e58b964d306 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:10.272550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 845/1660] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:11.621957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2b6d-12f0-4b37-b98d-5c1865d571f5 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:11.623218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2b6d-12f0-4b37-b98d-5c1865d571f5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:11.623651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2b6d-12f0-4b37-b98d-5c1865d571f5 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:11.625838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 816/1661] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 14:57:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:11.637939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536092fd-501b-40d3-bcfb-3f38f96ef03a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:11.638343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536092fd-501b-40d3-bcfb-3f38f96ef03a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:11.638446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536092fd-501b-40d3-bcfb-3f38f96ef03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:11.686054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-536092fd-501b-40d3-bcfb-3f38f96ef03a None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:11.687414 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-536092fd-501b-40d3-bcfb-3f38f96ef03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:11.707687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 846/1662] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 14:57:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:11.718980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c908b3-6052-4ecc-a7bb-33c10d2dcfd3 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:11.719888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c908b3-6052-4ecc-a7bb-33c10d2dcfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:11.720183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c908b3-6052-4ecc-a7bb-33c10d2dcfd3 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:11.723398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 817/1663] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 14:57:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:11.733013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3e758-b874-443a-9a84-34fa776e3a1b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:11.733329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3e758-b874-443a-9a84-34fa776e3a1b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:11.733695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3e758-b874-443a-9a84-34fa776e3a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:11.744242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:11.771757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-91e3e758-b874-443a-9a84-34fa776e3a1b None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:11.773937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-91e3e758-b874-443a-9a84-34fa776e3a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:11.800246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 847/1664] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:57:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:11.818916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c908b3-6052-4ecc-a7bb-33c10d2dcfd3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:11.837211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c908b3-6052-4ecc-a7bb-33c10d2dcfd3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:11.848827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402738c5-e5a1-4bc1-8488-6d9dbd7da5dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:11.849887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402738c5-e5a1-4bc1-8488-6d9dbd7da5dc service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:11.850361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402738c5-e5a1-4bc1-8488-6d9dbd7da5dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:11.851502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402738c5-e5a1-4bc1-8488-6d9dbd7da5dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:11.882989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402738c5-e5a1-4bc1-8488-6d9dbd7da5dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:11.884856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402738c5-e5a1-4bc1-8488-6d9dbd7da5dc service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:11.918173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 818/1665] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:13.176210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3e758-b874-443a-9a84-34fa776e3a1b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:13.324824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3e758-b874-443a-9a84-34fa776e3a1b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:13.341876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:13.342840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:13.343281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:13.343859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] PATH_INFO: `/v3/users/1089c2be3fa14a1890f901d27576f4f0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:13.346775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1089c2be3fa14a1890f901d27576f4f0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:13.347988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:57:13.347988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 09 14:57:13.347988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: Sep 09 14:57:13.351627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:13.515151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1089c2be3fa14a1890f901d27576f4f0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:13.535107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1089c2be3fa14a1890f901d27576f4f0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:13.593976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1089c2be3fa14a1890f901d27576f4f0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:13.594895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1089c2be3fa14a1890f901d27576f4f0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:13.612443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1089c2be3fa14a1890f901d27576f4f0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:13.623143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 848/1666] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:13 2022] DELETE /identity/v3/users/1089c2be3fa14a1890f901d27576f4f0 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:13.628458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:13.628940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:13.629413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:13.673056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:13.675455 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:13.905790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 819/1667] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:14.129472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:14.131076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:14.378424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ada28d-0fc0-4721-bb2c-a5e1ae2fb073 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:14.398322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dd784-6e4f-48af-9007-009720be95cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:14.399147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd784-6e4f-48af-9007-009720be95cf service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:14.399528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd784-6e4f-48af-9007-009720be95cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:14.400163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd784-6e4f-48af-9007-009720be95cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:14.441088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dd784-6e4f-48af-9007-009720be95cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:14.446198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dd784-6e4f-48af-9007-009720be95cf service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:14.494411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 849/1668] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:16.281790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:16.287672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:16.426448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:16.471097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fd10-9fc4-4882-8bf5-c6cb22174920 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:16.475519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e694157-2fc3-46e0-a8ed-46d309a1ab1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:16.476133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e694157-2fc3-46e0-a8ed-46d309a1ab1f service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:16.476411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e694157-2fc3-46e0-a8ed-46d309a1ab1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:16.476678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e694157-2fc3-46e0-a8ed-46d309a1ab1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:16.481043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9e694157-2fc3-46e0-a8ed-46d309a1ab1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:16.649462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e694157-2fc3-46e0-a8ed-46d309a1ab1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:16.651727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e694157-2fc3-46e0-a8ed-46d309a1ab1f service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:16.691998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 820/1669] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:17.091382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:17.091872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:17.091945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:17.182961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] MFA Rules not processed for user `1e123765a8724a89863d74221001dcd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:17.187596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:17.203914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:17.398087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:17.398631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 850/1670] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:17.399061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:17.399474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:17.472818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] MFA Rules not processed for user `1e123765a8724a89863d74221001dcd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:17.474103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:17.512289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 821/1671] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:17.526411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:17.534042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:17.792278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11f02a-f511-433c-938a-87ca9af9e4be None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:17.812479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:17.813362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:17.813766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:17.814203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:17.832615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:17.834693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:17.865579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 851/1672] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:17.878625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:17.880217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:17.928858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0af35e-fcf6-4516-8aee-dedf3791c2f6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:17.957929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:17.958734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:17.959011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:17.959282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] PATH_INFO: `/v3/projects/571292c3d1e94192b8a841cc4df40549` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:17.960699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=571292c3d1e94192b8a841cc4df40549)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:17.961745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 14:57:17.961745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Sep 09 14:57:17.961745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: Sep 09 14:57:17.965620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:18.085298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '571292c3d1e94192b8a841cc4df40549', 'request_id': 'req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.113185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '571292c3d1e94192b8a841cc4df40549', 'request_id': 'req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.121877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '571292c3d1e94192b8a841cc4df40549', 'request_id': 'req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.154774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] The token cache is being invalidate because project 571292c3d1e94192b8a841cc4df40549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:18.155321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.156403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7232e04e-30cb-4c5f-b70f-4dd4932f45f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.160119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 822/1673] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:17 2022] DELETE /identity/v3/projects/571292c3d1e94192b8a841cc4df40549 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:18.165063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:18.184752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44dc4a7-9e83-4c70-a6ca-d2d94ec037d7 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:18.366453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:18.367424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:18.367914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:18.368370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] PATH_INFO: `/v3/users/68a16b43aad7402ebcb7382f46e1cc52` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:18.369943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a16b43aad7402ebcb7382f46e1cc52)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:18.374859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:18.494902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:18.514772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a16b43aad7402ebcb7382f46e1cc52'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.525718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a16b43aad7402ebcb7382f46e1cc52'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.549961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a16b43aad7402ebcb7382f46e1cc52'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.550762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a16b43aad7402ebcb7382f46e1cc52'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.571558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-015d3af0-0afa-403a-af12-e26a3ba3d680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a16b43aad7402ebcb7382f46e1cc52'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:18.584380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 852/1674] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:18 2022] DELETE /identity/v3/users/68a16b43aad7402ebcb7382f46e1cc52 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:18.718871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82fbf1-aa28-4fae-9cf2-f2437eac5de5 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:18.719394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82fbf1-aa28-4fae-9cf2-f2437eac5de5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:18.719394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82fbf1-aa28-4fae-9cf2-f2437eac5de5 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:18.720764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 823/1675] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 14:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:57:18.727215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:18.728084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:18.728528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:18.804163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] MFA Rules not processed for user `f9edd8cf448e42fd8e888c8f531fa70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:18.816310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:18.989570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82fbf1-aa28-4fae-9cf2-f2437eac5de5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:18.992701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3d82fbf1-aa28-4fae-9cf2-f2437eac5de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:19.003764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 853/1676] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:57:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:19.020163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82fbf1-aa28-4fae-9cf2-f2437eac5de5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:19.030629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:19.031660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:19.033782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:19.034477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:19.049781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:19.052059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:19.064239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 824/1677] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:20.127746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:20.129887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:20.299542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037d77-8095-421f-8f85-6e6d8671fa70 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:20.301499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:20.303081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:20.314433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:20.315560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:20.315560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:20.328383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:20.334229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:20.366880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:20.368768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:20.422507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 854/1678] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:20.490444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f273592-ee55-430d-8dc2-b79774e2622e service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:20.499266 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:20.500231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:20.501766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:20.502597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] PATH_INFO: `/v3/users/4c0b5403b8814a68b4615a787f79c32f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:20.507520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0b5403b8814a68b4615a787f79c32f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:20.514390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:20.520705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:20.544766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:20.546331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:20.585842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3593f-e90e-4f15-ae2d-b7a601ff78e7 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:20.597982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:20.601660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:20.601980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:20.602267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] PATH_INFO: `/v3/users/9ef5ed0139254c1981166c806f70d036` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:20.618589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef5ed0139254c1981166c806f70d036)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:20.625753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:20.656175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0b5403b8814a68b4615a787f79c32f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.677258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0b5403b8814a68b4615a787f79c32f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.686838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0b5403b8814a68b4615a787f79c32f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.701327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0b5403b8814a68b4615a787f79c32f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.702461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0b5403b8814a68b4615a787f79c32f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.719800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 825/1679] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:20 2022] DELETE /identity/v3/users/4c0b5403b8814a68b4615a787f79c32f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:20.721010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:20.723845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:20.728021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:20.774540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef5ed0139254c1981166c806f70d036'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.797593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef5ed0139254c1981166c806f70d036'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.871154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef5ed0139254c1981166c806f70d036'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.872611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef5ed0139254c1981166c806f70d036'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.898886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dc2217ab-27f5-4381-9951-9c5cf8b7b7a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef5ed0139254c1981166c806f70d036'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:20.917803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 855/1680] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:20 2022] DELETE /identity/v3/users/9ef5ed0139254c1981166c806f70d036 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:20.920359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efe82-d916-4fae-a40d-fc846f06f3ab None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:20.920749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efe82-d916-4fae-a40d-fc846f06f3ab None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:20.921084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32efe82-d916-4fae-a40d-fc846f06f3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:20.992921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6591-8403-465c-8220-d6d3b52ae3a8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:21.007216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b32efe82-d916-4fae-a40d-fc846f06f3ab None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:21.008465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b32efe82-d916-4fae-a40d-fc846f06f3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:21.135563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 856/1681] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:21.138166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:21.138587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:21.138996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:21.179443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:21.184025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:21.204597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 857/1682] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:21.207818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:21.209702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:21.223841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:21.234166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:21.234813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:21.235474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] PATH_INFO: `/v3/projects/c1fe70fb7c2a4e4ca6be0e60838a8837` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:21.237538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fe70fb7c2a4e4ca6be0e60838a8837)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:21.240207 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:21.330703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:21.345079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881f1f8-7d04-402d-a9ee-25606799f944 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:21.390997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fe70fb7c2a4e4ca6be0e60838a8837', 'request_id': 'req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.419556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fe70fb7c2a4e4ca6be0e60838a8837', 'request_id': 'req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.437422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:21.439163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:21.439672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:21.440142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:21.448351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fe70fb7c2a4e4ca6be0e60838a8837', 'request_id': 'req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.497030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] The token cache is being invalidate because project c1fe70fb7c2a4e4ca6be0e60838a8837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:21.497695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.498913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.500567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:21.502077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 826/1683] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:20 2022] DELETE /identity/v3/projects/c1fe70fb7c2a4e4ca6be0e60838a8837 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:21.503039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:21.504664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:21.526776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e752f50-4afa-4c5f-a4b9-78f937ecf69f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:21.533305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b845970-728c-4d80-8df3-f3672f059ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:21.534118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b845970-728c-4d80-8df3-f3672f059ccc service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:21.534762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b845970-728c-4d80-8df3-f3672f059ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:21.536434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b845970-728c-4d80-8df3-f3672f059ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:21.541073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8b845970-728c-4d80-8df3-f3672f059ccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:21.629936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:21.632417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:21.651883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 858/1684] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:21.652906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:21.654889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:21.673894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b845970-728c-4d80-8df3-f3672f059ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:21.675737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b845970-728c-4d80-8df3-f3672f059ccc service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:21.680670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc2924-6b60-483b-b17b-3c35f2660bc6 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:21.696359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 827/1685] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:21.697281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:21.697930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:21.698516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:21.698684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:21.698785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:21.699136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] PATH_INFO: `/v3/users/d86692123ae8418a8126d54291469049` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:21.699265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:21.701720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86692123ae8418a8126d54291469049)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:21.706022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:21.745001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] MFA Rules not processed for user `cd6c40dd492e40599324ef05011895f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:21.746635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:21.817766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86692123ae8418a8126d54291469049'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.831241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86692123ae8418a8126d54291469049'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.862651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86692123ae8418a8126d54291469049'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.863423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86692123ae8418a8126d54291469049'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.882976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2dd38e5c-9d6c-4c51-badf-b869d319869e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86692123ae8418a8126d54291469049'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:21.910366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 859/1686] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:21 2022] DELETE /identity/v3/users/d86692123ae8418a8126d54291469049 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:21.929733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 828/1687] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:21.943372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:21.943671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:21.943941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:21.980464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] MFA Rules not processed for user `cd6c40dd492e40599324ef05011895f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:21.981734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:22.147057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 860/1688] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:22.162185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:22.164254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:22.254379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:22.328033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1eee9-3fc5-4db1-aeae-579347d9a911 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:22.447303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:22.448023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:22.448174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:22.448434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:22.459051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:22.460972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:22.472814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 829/1689] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:22.816659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:22.830806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:22.904771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:22.905928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:22.907404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:23.002681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2139b-dcd6-48ff-99e8-3ea085773ce3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:23.021940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:23.022596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:23.022871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:23.023144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] PATH_INFO: `/v3/projects/6722a8ce619f44ec8f63cb969ae8f460` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:23.025946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6722a8ce619f44ec8f63cb969ae8f460)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:23.029353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:23.077822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fc6d4-0286-4b47-b785-a477906e8367 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:23.090668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:23.091327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:23.091604 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:23.091877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] PATH_INFO: `/v3/projects/dbf5912f0c324ba3bb5312868183c630` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:23.099232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf5912f0c324ba3bb5312868183c630)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:23.101404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:23.119981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6722a8ce619f44ec8f63cb969ae8f460', 'request_id': 'req-80a96d95-586c-43c6-838f-d4474cc9f851'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.132610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6722a8ce619f44ec8f63cb969ae8f460', 'request_id': 'req-80a96d95-586c-43c6-838f-d4474cc9f851'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.149549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6722a8ce619f44ec8f63cb969ae8f460', 'request_id': 'req-80a96d95-586c-43c6-838f-d4474cc9f851'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.194376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] The token cache is being invalidate because project 6722a8ce619f44ec8f63cb969ae8f460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:23.194690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.196313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-80a96d95-586c-43c6-838f-d4474cc9f851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.201039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 861/1690] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:22 2022] DELETE /identity/v3/projects/6722a8ce619f44ec8f63cb969ae8f460 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:23.205015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa94c5-3252-4d72-9359-ed84f6ac29e7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:23.205417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa94c5-3252-4d72-9359-ed84f6ac29e7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:23.211212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa94c5-3252-4d72-9359-ed84f6ac29e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:23.272324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf5912f0c324ba3bb5312868183c630', 'request_id': 'req-5a206d78-403f-4a00-9d08-5a34e51db2a1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.282105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf5912f0c324ba3bb5312868183c630', 'request_id': 'req-5a206d78-403f-4a00-9d08-5a34e51db2a1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.290733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf5912f0c324ba3bb5312868183c630', 'request_id': 'req-5a206d78-403f-4a00-9d08-5a34e51db2a1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.297708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:23.304860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-10fa94c5-3252-4d72-9359-ed84f6ac29e7 None None] MFA Rules not processed for user `770105b06e084179a819a54d28b70cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:23.306256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-10fa94c5-3252-4d72-9359-ed84f6ac29e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:23.322746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] The token cache is being invalidate because project dbf5912f0c324ba3bb5312868183c630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:23.323823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.324366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.329847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 830/1691] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:22 2022] DELETE /identity/v3/projects/dbf5912f0c324ba3bb5312868183c630 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:23.329847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:23.330707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:23.491055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 862/1692] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:23.491055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc67fec-0472-49bc-b54c-ac2bddc39ba9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:23.491055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc67fec-0472-49bc-b54c-ac2bddc39ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:23.491055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc67fec-0472-49bc-b54c-ac2bddc39ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:23.546405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206d78-403f-4a00-9d08-5a34e51db2a1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:23.554836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:23.555298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:23.555584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:23.555888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] PATH_INFO: `/v3/projects/2ef9057fcdbb4cef93ccf051e4ef2231` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:23.557221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef9057fcdbb4cef93ccf051e4ef2231)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:23.561134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:23.568297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5dc67fec-0472-49bc-b54c-ac2bddc39ba9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:23.573336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5dc67fec-0472-49bc-b54c-ac2bddc39ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:23.611416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 863/1693] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:23.612706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a7df-2230-4eea-a41a-8ccf59c50b32 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:23.612993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a7df-2230-4eea-a41a-8ccf59c50b32 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:23.613299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a7df-2230-4eea-a41a-8ccf59c50b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:23.692357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d062a7df-2230-4eea-a41a-8ccf59c50b32 None None] MFA Rules not processed for user `3c0c848f7858486c91773b34047d8aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:23.693664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d062a7df-2230-4eea-a41a-8ccf59c50b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:23.731078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef9057fcdbb4cef93ccf051e4ef2231', 'request_id': 'req-fc78c7b8-e349-46b4-aaff-40214c271061'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.754981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef9057fcdbb4cef93ccf051e4ef2231', 'request_id': 'req-fc78c7b8-e349-46b4-aaff-40214c271061'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.784641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:23.804328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef9057fcdbb4cef93ccf051e4ef2231', 'request_id': 'req-fc78c7b8-e349-46b4-aaff-40214c271061'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.855780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 864/1694] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:23.857379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc604-2791-4fdd-b67e-12c6c6f1e567 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:23.857821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc604-2791-4fdd-b67e-12c6c6f1e567 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:23.859128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc604-2791-4fdd-b67e-12c6c6f1e567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:23.867254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] The token cache is being invalidate because project 2ef9057fcdbb4cef93ccf051e4ef2231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:23.867668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.874593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:23.885986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 831/1695] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:23 2022] DELETE /identity/v3/projects/2ef9057fcdbb4cef93ccf051e4ef2231 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:23.889743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:23.892240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:23.914435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5aadc604-2791-4fdd-b67e-12c6c6f1e567 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:23.918375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5aadc604-2791-4fdd-b67e-12c6c6f1e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:24.137316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 865/1696] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:24.139062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c63f-4e64-446d-8e72-8ba05008b360 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:24.139774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c63f-4e64-446d-8e72-8ba05008b360 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:24.140252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c63f-4e64-446d-8e72-8ba05008b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:24.184316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78c7b8-e349-46b4-aaff-40214c271061 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:24.203742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5f006-cb62-4411-ad36-e80c0b3a9338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:24.204388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5f006-cb62-4411-ad36-e80c0b3a9338 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:24.204677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5f006-cb62-4411-ad36-e80c0b3a9338 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:24.204940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5f006-cb62-4411-ad36-e80c0b3a9338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:24.211396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9fa5f006-cb62-4411-ad36-e80c0b3a9338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:24.239970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6bc7c63f-4e64-446d-8e72-8ba05008b360 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:24.244588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6bc7c63f-4e64-446d-8e72-8ba05008b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:24.256123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:24.266520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 866/1697] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:24.268208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c3b38-3ddc-4602-8632-8b1c6aff8ef3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:24.268752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c3b38-3ddc-4602-8632-8b1c6aff8ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:24.269515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c3b38-3ddc-4602-8632-8b1c6aff8ef3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:24.272542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 867/1698] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 14:57:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:24.273728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c3b38-3ddc-4602-8632-8b1c6aff8ef3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:24.278407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c29c3b38-3ddc-4602-8632-8b1c6aff8ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:24.344961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c3b38-3ddc-4602-8632-8b1c6aff8ef3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:24.368731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:24.372422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:24.383137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:24.384902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] PATH_INFO: `/v3/users/70b106efba964581a87932573ea17b26` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:24.388336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b106efba964581a87932573ea17b26)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:24.393071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:24.416242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5f006-cb62-4411-ad36-e80c0b3a9338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:24.418538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5f006-cb62-4411-ad36-e80c0b3a9338 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:24.444011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 832/1699] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:24.461663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:24.461954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:24.462247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:24.517038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b106efba964581a87932573ea17b26'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:24.540990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b106efba964581a87932573ea17b26'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:24.603720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b106efba964581a87932573ea17b26'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:24.609320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b106efba964581a87932573ea17b26'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:24.622126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:24.627313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:24.639103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17c8d71b-9cba-41d2-908c-9f86b351c7b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b106efba964581a87932573ea17b26'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:24.653799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 868/1700] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:24 2022] DELETE /identity/v3/users/70b106efba964581a87932573ea17b26 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:24.656522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129750cc-ac8b-4484-9068-beaa5d93c559 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:24.656811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129750cc-ac8b-4484-9068-beaa5d93c559 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:24.657091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129750cc-ac8b-4484-9068-beaa5d93c559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:24.725511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-129750cc-ac8b-4484-9068-beaa5d93c559 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:24.726838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-129750cc-ac8b-4484-9068-beaa5d93c559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:24.826505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:24.837238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 833/1701] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:24.838264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:24.842126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:24.881682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29b296-52cf-4d63-96ba-3e6b2548478d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:24.924131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 869/1702] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:24.940965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5f578-6df5-480c-a27a-fc8705e82211 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:24.941418 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5f578-6df5-480c-a27a-fc8705e82211 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:24.941701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5f578-6df5-480c-a27a-fc8705e82211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:24.986185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6be5f578-6df5-480c-a27a-fc8705e82211 None None] MFA Rules not processed for user `f9edd8cf448e42fd8e888c8f531fa70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:24.989871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6be5f578-6df5-480c-a27a-fc8705e82211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:25.066533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:25.067176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:25.067462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:25.067740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:25.070578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:25.070910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:25.076811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:25.111809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 834/1703] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:25.112816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:25.120670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:25.157810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 870/1704] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:25.158831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5f578-6df5-480c-a27a-fc8705e82211 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:25.189722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5f578-6df5-480c-a27a-fc8705e82211 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:25.207964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:25.208610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:25.209042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:25.209357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:25.210030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:25.210338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:25.211989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:25.227181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:25.251947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 871/1705] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:25.251947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:25.273485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989b98f-4337-4aae-81f2-ba27db720324 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:25.288525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:25.289505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:25.289855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:25.290139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:25.291455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:25.296878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:25.364836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2bf00-0cb8-4a8f-a126-1abc14c7e48d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:25.381586 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:25.382330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:25.382676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:25.382955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:25.388224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:25.389914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 872/1706] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:25 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:25.390905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:25.394801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:25.552768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:25.555153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:25.587715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 835/1707] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:25.592590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:25.599036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:25.629063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e2595-9d79-43d9-a07a-67f5bc69b24e service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:25.719332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f536364-865f-48ad-bafe-472841528c36 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:25.735139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:25.737081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:25.743266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:25.743462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:25.746686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:25.747386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:25.747678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:25.747952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:25.802292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:25.803906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:25.818346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 836/1708] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:25.819057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:25.916555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:26.021844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:26.023593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:26.047456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 873/1709] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 14:57:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 14:57:26.051204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:26.080782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3256e-d6ad-4f6c-8866-75e35d9dc906 service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:26.081006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba893213-0728-4902-8c49-3ec7b4e30c3d service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:26.108822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:26.109039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:26.112686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:26.113062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:26.113675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:26.114108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:26.114306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:26.114980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:26.115317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:26.116087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:26.123223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:26.126185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:26.170830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:26.201281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 874/1710] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:26 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:26.205357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:26.221254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 837/1711] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:25 2022] POST /identity/v3/projects => generated 403 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:26.225838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:26.233932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c6fe1-f7f2-4d5c-a946-40b918becae3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:26.385495 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0610-d932-4876-9a6b-9b4aeb45bb34 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:26.407539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:26.408630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:26.409419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:26.410596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:26.410596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:26.412696 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:26.472754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:26.480116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:26.484915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 838/1712] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:26 2022] POST /identity/v3/users => generated 302 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:26.486687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:26.499771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:26.499911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:26.500208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:26.500930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:26.520085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:26.562087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b846bb2-e17e-46dc-8755-bba097ff2124 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:26.595769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:26.596611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:26.596719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:26.596978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] PATH_INFO: `/v3/projects/31669bdc8f4a4b6c9bfc3b4d018333a2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:26.606578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31669bdc8f4a4b6c9bfc3b4d018333a2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:26.610694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:26.623092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 875/1713] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:26.637820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:26.684035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d07792-e05b-4852-85fb-3c543a4cc92f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:26.732475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:26.733023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:26.741901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:26.742395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:26.743121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:26.755033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:26.822823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 876/1714] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:26.827160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:26.851338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31669bdc8f4a4b6c9bfc3b4d018333a2', 'request_id': 'req-84cae961-f521-426a-b7dc-ea70d7d829f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:26.880632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8853087-9f00-4286-a6d9-5a3a1aedacbb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:26.881307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31669bdc8f4a4b6c9bfc3b4d018333a2', 'request_id': 'req-84cae961-f521-426a-b7dc-ea70d7d829f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:26.906383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31669bdc8f4a4b6c9bfc3b4d018333a2', 'request_id': 'req-84cae961-f521-426a-b7dc-ea70d7d829f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:26.991673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] The token cache is being invalidate because project 31669bdc8f4a4b6c9bfc3b4d018333a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:26.992016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:26.999772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-84cae961-f521-426a-b7dc-ea70d7d829f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:27.013352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 839/1715] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:26 2022] DELETE /identity/v3/projects/31669bdc8f4a4b6c9bfc3b4d018333a2 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:27.014181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:27.014570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:27.014782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:27.122592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] MFA Rules not processed for user `f9edd8cf448e42fd8e888c8f531fa70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:27.125519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:27.235250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:27.237955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:27.238638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:27.238638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:27.238846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 840/1716] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Sep 9 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:27.239608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:27.240183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:27.241046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:27.249395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:27.273396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4047b-4371-406a-8ff2-e834fa069618 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:27.282559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:27.283218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:27.283441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:27.283894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:27.284932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:27.287492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:27.307397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 841/1717] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:27.307397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:27.314171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 877/1718] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:27.316902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:27.342937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:27.366937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:27.374110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9662c29-f36f-4d81-97f3-63911417c61d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:27.391673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:27.392531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:27.392800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:27.393152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] PATH_INFO: `/v3/users/84d213651aec43928ac23ed2726ec3e0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:27.396184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f036f8a-5b33-4176-8859-b1cc980c7165 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:27.397036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d213651aec43928ac23ed2726ec3e0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:27.399501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:27.421788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:27.421788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:27.421788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:27.421788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] PATH_INFO: `/v3/projects/7b907e5fa35a49ad9ba11718211a3275/users/879a7af9d8384b469ad8b684066d0ec9/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:27.501170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b907e5fa35a49ad9ba11718211a3275, user_id=879a7af9d8384b469ad8b684066d0ec9, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:27.504170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:27.524382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d213651aec43928ac23ed2726ec3e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:27.545092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 878/1719] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:27 2022] PUT /identity/v3/projects/7b907e5fa35a49ad9ba11718211a3275/users/879a7af9d8384b469ad8b684066d0ec9/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:27.550437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:27.557509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:27.560551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d213651aec43928ac23ed2726ec3e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:27.592808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d213651aec43928ac23ed2726ec3e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:27.610480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d213651aec43928ac23ed2726ec3e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:27.612027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d213651aec43928ac23ed2726ec3e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:27.617681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:27.621981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 842/1720] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:27 2022] DELETE /identity/v3/users/84d213651aec43928ac23ed2726ec3e0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:27.623740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:27.634099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:27.744212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf816db-0b09-4dd1-8452-a6611e93485f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:27.909439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad8d6b-450d-499a-9025-c7e48a744b56 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:27.924233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a27a9-13ac-4f97-9fe5-2b731f022cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:27.924876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a27a9-13ac-4f97-9fe5-2b731f022cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:27.925155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a27a9-13ac-4f97-9fe5-2b731f022cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:27.925451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a27a9-13ac-4f97-9fe5-2b731f022cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:27.927919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-366a27a9-13ac-4f97-9fe5-2b731f022cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:27.937939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a83acf-2182-44b4-918c-f955ff043a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:27.938658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a83acf-2182-44b4-918c-f955ff043a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:27.938939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a83acf-2182-44b4-918c-f955ff043a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:27.940639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a83acf-2182-44b4-918c-f955ff043a19 admin admin] PATH_INFO: `/v3/projects/658d1ce2094543edb865bceee0097e57/users/04a3ae870c5640de96ee54315ec815d1/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:27.968748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a27a9-13ac-4f97-9fe5-2b731f022cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:27.970588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a27a9-13ac-4f97-9fe5-2b731f022cfb service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:28.005278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a83acf-2182-44b4-918c-f955ff043a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658d1ce2094543edb865bceee0097e57, user_id=04a3ae870c5640de96ee54315ec815d1, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:28.014495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a83acf-2182-44b4-918c-f955ff043a19 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:28.021142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 843/1721] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:28.022656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b094e0-bcfb-4e03-87a1-961e854f6986 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:28.022952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b094e0-bcfb-4e03-87a1-961e854f6986 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:28.023226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b094e0-bcfb-4e03-87a1-961e854f6986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:28.051254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 879/1722] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:27 2022] PUT /identity/v3/projects/658d1ce2094543edb865bceee0097e57/users/04a3ae870c5640de96ee54315ec815d1/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:28.062759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:28.063321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:28.063656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:28.164626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d7b094e0-bcfb-4e03-87a1-961e854f6986 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:28.168197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] MFA Rules not processed for user `04a3ae870c5640de96ee54315ec815d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:28.169466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:28.171639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d7b094e0-bcfb-4e03-87a1-961e854f6986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:28.484635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 844/1723] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:28.487471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d7c0-fb0c-4e18-a6c1-35a1b8cdd22f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:28.487471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d7c0-fb0c-4e18-a6c1-35a1b8cdd22f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:28.497323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d7c0-fb0c-4e18-a6c1-35a1b8cdd22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:28.506038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 880/1724] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:28.515301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:28.522721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:28.607371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711ff9-c319-42ce-b0ed-2cd4459364f7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:28.613902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-52b6d7c0-fb0c-4e18-a6c1-35a1b8cdd22f None None] MFA Rules not processed for user `cd6c40dd492e40599324ef05011895f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:28.615213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-52b6d7c0-fb0c-4e18-a6c1-35a1b8cdd22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:28.637170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:28.638042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:28.641526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:28.641843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:28.642523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:28.651659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:28.679045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:28.829128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 881/1725] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:28 2022] POST /identity/v3/projects => generated 381 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:28.830217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:28.867087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:28.971622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 845/1726] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:28.974500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d7c0-fb0c-4e18-a6c1-35a1b8cdd22f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:29.027957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342aca3-1d0c-48c2-bd66-5001b7229491 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:29.030657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d7c0-fb0c-4e18-a6c1-35a1b8cdd22f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:29.044496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:29.047568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:29.048028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:29.049731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:29.262830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:29.263477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:29.263764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:29.264041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:29.264691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:29.275974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:29.286554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:29.288947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:29.310738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 882/1727] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:29.311637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:29.313220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:29.330567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 846/1728] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:28 2022] POST /identity/v3/users => generated 297 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:29.330567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:29.332991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:29.351792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3895d2-cc50-4919-88e0-fe7a08a3077a service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:29.360250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939eca-1bb2-418f-96de-66857964fb53 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:29.365314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:29.366286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:29.366576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:29.366856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] PATH_INFO: `/v3/users/fa66856f48284deca2b7f7b0e729c720` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:29.368669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa66856f48284deca2b7f7b0e729c720)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:29.371545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:29.375802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:29.376778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:29.376778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:29.377037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] PATH_INFO: `/v3/projects/405c1623ad1a42b38fbab864d1feda77` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:29.378731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] RBAC: Authorizing `identity:delete_project(project_id=405c1623ad1a42b38fbab864d1feda77)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:29.382904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:29.497352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa66856f48284deca2b7f7b0e729c720'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.503704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405c1623ad1a42b38fbab864d1feda77', 'request_id': 'req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.512823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405c1623ad1a42b38fbab864d1feda77', 'request_id': 'req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.519875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405c1623ad1a42b38fbab864d1feda77', 'request_id': 'req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.540819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] The token cache is being invalidate because project 405c1623ad1a42b38fbab864d1feda77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:29.541864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.543295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.550268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 847/1729] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:29 2022] DELETE /identity/v3/projects/405c1623ad1a42b38fbab864d1feda77 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:29.553621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:29.560572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:29.564583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa66856f48284deca2b7f7b0e729c720'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.652351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa66856f48284deca2b7f7b0e729c720'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.660540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa66856f48284deca2b7f7b0e729c720'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.704995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4a2543b-aa76-4243-a697-eeadab186bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa66856f48284deca2b7f7b0e729c720'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:29.736287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 883/1730] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:29 2022] DELETE /identity/v3/users/fa66856f48284deca2b7f7b0e729c720 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:29.742202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4ca2-541e-425c-9470-b518a689a495 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:29.742765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4ca2-541e-425c-9470-b518a689a495 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:29.744451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4ca2-541e-425c-9470-b518a689a495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:29.784653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:29.833248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4e4c4ca2-541e-425c-9470-b518a689a495 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:29.842338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4e4c4ca2-541e-425c-9470-b518a689a495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:29.890502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e23a7e-65ff-4fc6-bee9-1b5fdbe2b91a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:29.957844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:30.064169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 884/1731] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:30.067998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77656-c0d5-4ecd-950a-6da8563eec7a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.068290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77656-c0d5-4ecd-950a-6da8563eec7a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.068564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77656-c0d5-4ecd-950a-6da8563eec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.141300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9ea77656-c0d5-4ecd-950a-6da8563eec7a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:30.145913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9ea77656-c0d5-4ecd-950a-6da8563eec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:30.153080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:30.153756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.154040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.154312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.163972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:30.165932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 885/1732] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:30.167088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.167378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.167675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.173693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:30.225070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 848/1733] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 672 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:30.226172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:30.238188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:30.253653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:30.262225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:30.264004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6aa94-6154-463a-a7c5-cfbbcb18cb44 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:30.272971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:30.273815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.274084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.274904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] PATH_INFO: `/v3/projects/debe98ba25a24934be06d95e854157f9/users/49c00c60f91c4252809d831beb9b2496/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.308747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 886/1734] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:30.309721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:30.312015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=debe98ba25a24934be06d95e854157f9, user_id=49c00c60f91c4252809d831beb9b2496, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:30.315142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:30.319841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:30.348337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 849/1735] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:30 2022] PUT /identity/v3/projects/debe98ba25a24934be06d95e854157f9/users/49c00c60f91c4252809d831beb9b2496/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:30.352106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:30.583075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1efac8-62f3-4a24-bd64-058ce25832ae None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:30.607839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab99a9d-ab6c-46fa-a82e-080b2c9b4ba3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:30.628251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:30.628892 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.629279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.629593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.630391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:30.630634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:30.632271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:30.689638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 850/1736] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:30.691236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:30.720183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece32c-b00f-46e8-aeb1-0ff1b90b0045 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:30.743667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:30.745520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.746937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.748220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.750827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:30.755430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:30.800958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 851/1737] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:30.810385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:30.847758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d432b5e-c2ad-4dfb-bda1-05213104a89d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:30.861833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:30.862556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.862989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.863252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.864056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:30.868816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:30.870753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:30.871983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:30.873760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:30.874343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:30.897930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:30.899587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:30.936007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 887/1738] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:30.938668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:30.972354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 852/1739] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:30 2022] POST /identity/v3/projects => generated 401 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:30.973937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:30.984958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34389edd-e021-49f6-a54f-9956915ec107 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:31.080396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:31.218910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ac258-8658-4d8c-b027-f31e2051614c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:31.226161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30411ce1-f59b-4edc-9a7e-f68ff50ea7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:31.226857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30411ce1-f59b-4edc-9a7e-f68ff50ea7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:31.227147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30411ce1-f59b-4edc-9a7e-f68ff50ea7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:31.227430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30411ce1-f59b-4edc-9a7e-f68ff50ea7f7 admin admin] PATH_INFO: `/v3/projects/debe98ba25a24934be06d95e854157f9/users/49c00c60f91c4252809d831beb9b2496/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:31.232597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:31.233245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:31.233531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:31.233807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:31.234451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:31.236398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:31.237911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30411ce1-f59b-4edc-9a7e-f68ff50ea7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=debe98ba25a24934be06d95e854157f9, user_id=49c00c60f91c4252809d831beb9b2496, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:31.244842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30411ce1-f59b-4edc-9a7e-f68ff50ea7f7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:31.280176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 888/1740] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:30 2022] PUT /identity/v3/projects/debe98ba25a24934be06d95e854157f9/users/49c00c60f91c4252809d831beb9b2496/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:31.292562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:31.294700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:31.295341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:31.335750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 853/1741] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:30 2022] POST /identity/v3/users => generated 301 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:31.337118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:31.344627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:31.357530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eddbe9-8373-4247-89d8-37235a191c50 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:31.371248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] MFA Rules not processed for user `49c00c60f91c4252809d831beb9b2496`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:31.385678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:31.736422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:31.737774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:31.739680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:31.741254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:31.743949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:31.749484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:31.800885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 889/1742] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:31.811123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:31.813710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:31.822016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 854/1743] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:31.827112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:31.856863 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e0da1-1302-441c-8b31-2284e80aa31d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:31.875409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:31.876711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:31.877219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:31.877837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:31.879157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:31.880977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:31.936854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 855/1744] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:31.940386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:31.970938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8a1e4-e3e0-43d9-bee7-ba519efa6efb admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:31.998505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18f53a-2efb-47e8-abb1-2f48537132e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:31.999645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18f53a-2efb-47e8-abb1-2f48537132e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:32.000330 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18f53a-2efb-47e8-abb1-2f48537132e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:32.001408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18f53a-2efb-47e8-abb1-2f48537132e4 admin admin] PATH_INFO: `/v3/projects/339ff606136346b7a5c81c533ef101b9/users/d09f5aee112041da83b745df8c4d536a/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:32.098838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18f53a-2efb-47e8-abb1-2f48537132e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=339ff606136346b7a5c81c533ef101b9, user_id=d09f5aee112041da83b745df8c4d536a, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:32.103843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18f53a-2efb-47e8-abb1-2f48537132e4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:32.161556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 856/1745] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:31 2022] PUT /identity/v3/projects/339ff606136346b7a5c81c533ef101b9/users/d09f5aee112041da83b745df8c4d536a/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:32.177521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9ab7a-4a85-4ef0-bf03-ae2929206197 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:32.178398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9ab7a-4a85-4ef0-bf03-ae2929206197 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:32.178763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cf156-bfd4-4f34-bfc6-764bd1f628a5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:32.179312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9ab7a-4a85-4ef0-bf03-ae2929206197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:32.327817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cff9ab7a-4a85-4ef0-bf03-ae2929206197 None None] MFA Rules not processed for user `d09f5aee112041da83b745df8c4d536a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:32.335972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cff9ab7a-4a85-4ef0-bf03-ae2929206197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:32.403004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:32.403888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:32.405502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:32.405798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:32.612796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:32.638176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 857/1746] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:32.641574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9ab7a-4a85-4ef0-bf03-ae2929206197 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:32.682986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:32.691860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:32.744568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 890/1747] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 933 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:33.001587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9ab7a-4a85-4ef0-bf03-ae2929206197 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:33.014484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:33.015185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:33.015459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:33.015724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:33.016433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:33.020582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:33.125540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 858/1748] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:32 2022] POST /identity/v3/projects => generated 397 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:33.126784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:33.139585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43518f9-34f5-496b-a16a-a94f0496b7d3 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:33.140528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:33.279495 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:33.283008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:33.317644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:33.318323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:33.320409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:33.320805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:33.323265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:33.328022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:33.391921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 891/1749] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:33 2022] POST /identity/v3/users => generated 305 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:57:33.399151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:33.438673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d06986-25fa-4052-b14c-3a1836734475 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:33.451762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:33.452676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:33.453113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:33.453566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:33.454335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:33.457089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:33.502029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7a5c1-a860-4163-9319-7630caf44501 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:33.522048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:33.522636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:33.522919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:33.523192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] PATH_INFO: `/v3/projects/fb1ef290da434a6e949d4c18d98c866e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:33.527654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1ef290da434a6e949d4c18d98c866e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:33.541347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:33.546238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 892/1750] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:33.549399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:33.593331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743b7c1-d313-47b0-8385-d3baf21cc8d1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:33.614829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:33.615692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:33.615970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:33.619064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] PATH_INFO: `/v3/projects/f0af9ab255c14c22b887f271c8faf173/users/d482d07bf6ca40b89ed40348817c8e84/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:33.674703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0af9ab255c14c22b887f271c8faf173, user_id=d482d07bf6ca40b89ed40348817c8e84, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:33.680599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:33.706098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1ef290da434a6e949d4c18d98c866e', 'request_id': 'req-825617de-cbe7-4d94-93f0-1fa9418b085f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:33.711238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 893/1751] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:33 2022] PUT /identity/v3/projects/f0af9ab255c14c22b887f271c8faf173/users/d482d07bf6ca40b89ed40348817c8e84/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:33.712213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:33.726666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1ef290da434a6e949d4c18d98c866e', 'request_id': 'req-825617de-cbe7-4d94-93f0-1fa9418b085f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:33.746648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1ef290da434a6e949d4c18d98c866e', 'request_id': 'req-825617de-cbe7-4d94-93f0-1fa9418b085f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:33.783650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] The token cache is being invalidate because project fb1ef290da434a6e949d4c18d98c866e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:33.783958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:33.787697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:33.793749 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 859/1752] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:33 2022] DELETE /identity/v3/projects/fb1ef290da434a6e949d4c18d98c866e => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:33.796732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:33.800638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:33.858073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:33.898073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5d3d-138c-49f7-913a-cbacb9290a5f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:33.916338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:33.916964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:33.917368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:33.917518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:33.924616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:33.931322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:33.958511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 894/1753] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:33.960656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:33.967207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:34.007981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af272-6cee-4ed5-ace7-9e5a25fc6de3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:34.032277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:34.032952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:34.039423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:34.040806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] PATH_INFO: `/v3/users/04a3ae870c5640de96ee54315ec815d1` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:34.046501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a3ae870c5640de96ee54315ec815d1)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:34.054568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:34.055728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825617de-cbe7-4d94-93f0-1fa9418b085f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:34.078901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:34.079742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:34.080129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:34.080534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] PATH_INFO: `/v3/users/755e26e7fcb944fc820e7a74cb3e4620` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:34.090141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755e26e7fcb944fc820e7a74cb3e4620)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:34.092372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:34.216909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755e26e7fcb944fc820e7a74cb3e4620'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.251131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755e26e7fcb944fc820e7a74cb3e4620'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.267842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755e26e7fcb944fc820e7a74cb3e4620'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.270727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a3ae870c5640de96ee54315ec815d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.279667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:34.299079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755e26e7fcb944fc820e7a74cb3e4620'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.299937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755e26e7fcb944fc820e7a74cb3e4620'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.309909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 860/1754] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:33 2022] DELETE /identity/v3/users/755e26e7fcb944fc820e7a74cb3e4620 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:34.313820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:34.314764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a3ae870c5640de96ee54315ec815d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.317834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:34.354713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a3ae870c5640de96ee54315ec815d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.357341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a3ae870c5640de96ee54315ec815d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.377099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-daa95748-abd8-48b1-bb62-9754ea95c7f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a3ae870c5640de96ee54315ec815d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:34.390984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 895/1755] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:33 2022] DELETE /identity/v3/users/04a3ae870c5640de96ee54315ec815d1 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:34.396794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324d64-3702-4ed6-ba09-995fb2c4f5ee None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:34.397127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324d64-3702-4ed6-ba09-995fb2c4f5ee None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:34.397445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324d64-3702-4ed6-ba09-995fb2c4f5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:34.474167 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bc324d64-3702-4ed6-ba09-995fb2c4f5ee None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:34.479433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bc324d64-3702-4ed6-ba09-995fb2c4f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:34.584226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f04099-0998-45ed-87cf-c5c82ce43f14 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:34.711158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 896/1756] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:34.729665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:34.832386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8071-0265-4e4c-b013-b331ed057ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:34.833149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8071-0265-4e4c-b013-b331ed057ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:34.837388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8071-0265-4e4c-b013-b331ed057ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:34.837823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8071-0265-4e4c-b013-b331ed057ea4 admin admin] PATH_INFO: `/v3/projects/f0af9ab255c14c22b887f271c8faf173/users/d482d07bf6ca40b89ed40348817c8e84/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:34.841403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc8071-0265-4e4c-b013-b331ed057ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0af9ab255c14c22b887f271c8faf173, user_id=d482d07bf6ca40b89ed40348817c8e84, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:34.845721 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc8071-0265-4e4c-b013-b331ed057ea4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:34.872081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 861/1757] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:34 2022] PUT /identity/v3/projects/f0af9ab255c14c22b887f271c8faf173/users/d482d07bf6ca40b89ed40348817c8e84/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:34.882801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c71b9-3922-4798-9648-58569463699e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:34.883348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c71b9-3922-4798-9648-58569463699e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:34.884056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c71b9-3922-4798-9648-58569463699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:34.900874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:34.901356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:34.901774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:34.977100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] MFA Rules not processed for user `879a7af9d8384b469ad8b684066d0ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:34.981889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:35.078110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-814c71b9-3922-4798-9648-58569463699e None None] MFA Rules not processed for user `d482d07bf6ca40b89ed40348817c8e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:35.086785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-814c71b9-3922-4798-9648-58569463699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:35.145902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 862/1758] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:35.146942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:35.154082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:35.274591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48ff21-7aa1-4b5c-8161-153460cac18a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:35.305063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:35.305911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:35.306278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:35.306382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 897/1759] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:34 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:35.307054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:35.307899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:35.311736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:35.312083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef052f-4a78-48b3-b0f3-8395998699ea None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:35.312366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef052f-4a78-48b3-b0f3-8395998699ea None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:35.312638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef052f-4a78-48b3-b0f3-8395998699ea None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:35.318401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 898/1760] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:57:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:35.319300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef052f-4a78-48b3-b0f3-8395998699ea None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:35.327565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1bef052f-4a78-48b3-b0f3-8395998699ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:35.384312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 863/1761] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:35 2022] POST /identity/v3/projects => generated 405 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:35.391735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:35.396248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:35.427191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef052f-4a78-48b3-b0f3-8395998699ea None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:35.436916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:35.509576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:35.521653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608def3-ceb1-48ed-a59c-96a92e0d7a1f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:35.521915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:35.522408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:35.522685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:35.522954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:35.528631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:35.529996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:35.531814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:35.532943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:35.608965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:35.610519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:35.623892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 899/1762] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:35.625089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:35.636500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec10ba-bc9e-433a-a6f9-b0b05ef76bed service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:35.643757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:35.644060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:35.645486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:35.645486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:35.645486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:35.647540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:35.676213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 900/1763] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:35 2022] POST /identity/v3/users => generated 309 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:35.677465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:35.690632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93881154-ce08-4539-99ae-c0790fb0faed admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:35.691819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:35.694123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:35.695939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:35.696699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:35.696908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:35.697247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:35.697997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:35.699540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:35.715204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 901/1764] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:35.725870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:35.731143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 864/1765] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:35 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:35.747962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906b7a-cd91-4ba1-aa86-a76331860a3e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:35.764007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:35.764888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:35.765273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:35.765627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] PATH_INFO: `/v3/projects/02743e3f4fb54c14b3e27331dbbdb9ae/users/68ef81de9b664b31989ba7e881013cfa/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:35.845499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02743e3f4fb54c14b3e27331dbbdb9ae, user_id=68ef81de9b664b31989ba7e881013cfa, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:35.854513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:35.887824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:35.916785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 902/1766] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:35 2022] PUT /identity/v3/projects/02743e3f4fb54c14b3e27331dbbdb9ae/users/68ef81de9b664b31989ba7e881013cfa/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:35.922432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd169fde-9a05-4276-8f50-6b7b23ecc986 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:35.923683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:36.228940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:36.233768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:36.234145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:36.237306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:36.243030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:36.269637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4baf7-3f8e-4eb6-afc1-4143acaaefd8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:36.282755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eda4ec-cbf1-468d-a71a-e8a955d589b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:36.284079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eda4ec-cbf1-468d-a71a-e8a955d589b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:36.284633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eda4ec-cbf1-468d-a71a-e8a955d589b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:36.285238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eda4ec-cbf1-468d-a71a-e8a955d589b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:36.286182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eda4ec-cbf1-468d-a71a-e8a955d589b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:36.294969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eda4ec-cbf1-468d-a71a-e8a955d589b3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:36.310789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:36.346444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 903/1767] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:36.348100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:36.350749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:36.351813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:36.427300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:36.429238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:36.432256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:36.439623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:36.467167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 865/1768] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:35 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:36.468236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:36.494941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46422721-9489-4f71-874f-1c72f75b469a service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:36.505835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98bed3-e23c-41eb-b0eb-80fd957e8890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:36.506622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98bed3-e23c-41eb-b0eb-80fd957e8890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:36.506956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98bed3-e23c-41eb-b0eb-80fd957e8890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:36.507246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98bed3-e23c-41eb-b0eb-80fd957e8890 admin admin] PATH_INFO: `/v3/projects/02743e3f4fb54c14b3e27331dbbdb9ae/users/68ef81de9b664b31989ba7e881013cfa/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:36.518358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98bed3-e23c-41eb-b0eb-80fd957e8890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02743e3f4fb54c14b3e27331dbbdb9ae, user_id=68ef81de9b664b31989ba7e881013cfa, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:36.522111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98bed3-e23c-41eb-b0eb-80fd957e8890 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:36.555977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 866/1769] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:36 2022] PUT /identity/v3/projects/02743e3f4fb54c14b3e27331dbbdb9ae/users/68ef81de9b664b31989ba7e881013cfa/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:36.617027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 904/1770] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:57:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:36.845837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:36.846407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:36.846407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:36.883679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:36.885139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:36.901926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] MFA Rules not processed for user `3c0c848f7858486c91773b34047d8aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:36.903754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:36.920069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966919d-9e35-48f4-a2bb-45d821d172f0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:37.038860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:37.156255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16a960-d986-4687-8b6b-a1b52235b8ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:37.157870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16a960-d986-4687-8b6b-a1b52235b8ac service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:37.158917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 867/1771] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:37.163881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16a960-d986-4687-8b6b-a1b52235b8ac service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:37.164416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16a960-d986-4687-8b6b-a1b52235b8ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:37.186073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16a960-d986-4687-8b6b-a1b52235b8ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:37.189388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16a960-d986-4687-8b6b-a1b52235b8ac service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:37.216483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 905/1772] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:57:37.556977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:37.559322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:37.741425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9c486-0853-49e2-a9f4-a4bf560b6b83 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:37.753131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:37.756026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:37.758205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:37.759075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] PATH_INFO: `/v3/projects/5eff1d3c3d4e43cc87b3390db10cdbb4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:37.763640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eff1d3c3d4e43cc87b3390db10cdbb4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:37.767049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:37.896504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eff1d3c3d4e43cc87b3390db10cdbb4', 'request_id': 'req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:37.954618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eff1d3c3d4e43cc87b3390db10cdbb4', 'request_id': 'req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:37.987473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eff1d3c3d4e43cc87b3390db10cdbb4', 'request_id': 'req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:38.020572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] The token cache is being invalidate because project 5eff1d3c3d4e43cc87b3390db10cdbb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:38.020861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:38.021799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1dcec7f1-a84d-4905-a5d9-fa1de9e4d1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:38.028058 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 868/1773] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:37 2022] DELETE /identity/v3/projects/5eff1d3c3d4e43cc87b3390db10cdbb4 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:38.028694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf90e3a-0eb7-4f95-a586-dbf69b833751 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:38.029167 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf90e3a-0eb7-4f95-a586-dbf69b833751 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:38.029282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf90e3a-0eb7-4f95-a586-dbf69b833751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:38.069885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3bf90e3a-0eb7-4f95-a586-dbf69b833751 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:38.071125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3bf90e3a-0eb7-4f95-a586-dbf69b833751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:38.242350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 906/1774] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:57:38.250905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96189c99-3ff3-4f8d-8123-e15ab7892a5c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:38.251189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96189c99-3ff3-4f8d-8123-e15ab7892a5c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:38.251535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96189c99-3ff3-4f8d-8123-e15ab7892a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:38.326170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-96189c99-3ff3-4f8d-8123-e15ab7892a5c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:38.328966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-96189c99-3ff3-4f8d-8123-e15ab7892a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:38.355049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 869/1775] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:38.359340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf90e3a-0eb7-4f95-a586-dbf69b833751 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:38.370659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf90e3a-0eb7-4f95-a586-dbf69b833751 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:38.390812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:38.391741 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:38.392164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:38.392578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:38.393404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:38.393849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:38.396490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:38.450131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96189c99-3ff3-4f8d-8123-e15ab7892a5c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:38.456991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 907/1776] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:38.463067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96189c99-3ff3-4f8d-8123-e15ab7892a5c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:38.472906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:38.473691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:38.474083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:38.474433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:38.475160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:38.479195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:38.575179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 870/1777] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:38 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:38.576535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:38.596941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:38.606538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3365e6c-179b-43a8-b7c7-7c83bafeba60 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:38.840277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:38.842163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:38.842163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:38.842163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:38.844856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:38.846935 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:38.897878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 908/1778] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:38 2022] POST /identity/v3/users => generated 291 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:38.903509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:38.938647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1440da-47ed-464c-b63d-d7bec6aaafcf admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:38.950896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:38.955063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:38.956077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:38.956451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:38.960891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:38.963885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:38.996467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 871/1779] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:39.000296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:39.014702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:39.019391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:39.037966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514e2a8-8a30-4bda-b718-a3c61a4cec2c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:39.075820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:39.077410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:39.077410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:39.077696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:39.078749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:39.080790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:39.107282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 909/1780] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:39.108774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:39.122160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f8aca-01ee-430a-bc6c-b127c5825f98 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:39.135976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9fa3d-710b-4be6-a04f-61c357a7ddfd admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:39.142798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:39.144992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:39.146283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:39.147245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] PATH_INFO: `/v3/projects/658d1ce2094543edb865bceee0097e57` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:39.152569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be99202-dcce-4247-9598-e809dcef12e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:39.152569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99202-dcce-4247-9598-e809dcef12e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:39.153857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=658d1ce2094543edb865bceee0097e57)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:39.155249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99202-dcce-4247-9598-e809dcef12e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:39.155535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99202-dcce-4247-9598-e809dcef12e8 admin admin] PATH_INFO: `/v3/projects/f03b52573d42491eb46c4a4315a9fc3b/users/6bdfc0fab3d34699ab16a78cbabff844/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:39.160782 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:39.192241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be99202-dcce-4247-9598-e809dcef12e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03b52573d42491eb46c4a4315a9fc3b, user_id=6bdfc0fab3d34699ab16a78cbabff844, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:39.197584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be99202-dcce-4247-9598-e809dcef12e8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:39.232804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 910/1781] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:39 2022] PUT /identity/v3/projects/f03b52573d42491eb46c4a4315a9fc3b/users/6bdfc0fab3d34699ab16a78cbabff844/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:39.243227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:39.243810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:39.244300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:39.332883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:39.376930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] MFA Rules not processed for user `6bdfc0fab3d34699ab16a78cbabff844`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:39.380373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:39.390244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658d1ce2094543edb865bceee0097e57', 'request_id': 'req-ac365a8f-15e5-405c-b3c2-46b0306343ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:39.416121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658d1ce2094543edb865bceee0097e57', 'request_id': 'req-ac365a8f-15e5-405c-b3c2-46b0306343ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:39.431146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658d1ce2094543edb865bceee0097e57', 'request_id': 'req-ac365a8f-15e5-405c-b3c2-46b0306343ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:39.462646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] The token cache is being invalidate because project 658d1ce2094543edb865bceee0097e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:39.463205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:39.466555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:39.471547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 872/1782] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:39 2022] DELETE /identity/v3/projects/658d1ce2094543edb865bceee0097e57 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:39.476298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:39.481591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:39.585697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 911/1783] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:57:39.591675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:39.594092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:39.773235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac365a8f-15e5-405c-b3c2-46b0306343ee admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:39.804179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:39.805166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:39.811213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:39.814047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] PATH_INFO: `/v3/users/49c00c60f91c4252809d831beb9b2496` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:39.817097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c00c60f91c4252809d831beb9b2496)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:39.830250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:39.834013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:39.958683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96070fa-3beb-49fd-9578-b11c63f25860 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:39.990620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:39.994579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:39.994579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:39.994902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:39.999451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:40.011446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:40.101887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c00c60f91c4252809d831beb9b2496'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:40.177738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 912/1784] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:39 2022] POST /identity/v3/projects => generated 379 bytes in 586 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:40.179295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:40.184517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c00c60f91c4252809d831beb9b2496'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:40.229272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c00c60f91c4252809d831beb9b2496'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:40.269601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c00c60f91c4252809d831beb9b2496'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:40.271623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-59bfb6bb-182d-4e7d-b546-12b47d1611d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c00c60f91c4252809d831beb9b2496'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:40.330420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 873/1785] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:39 2022] DELETE /identity/v3/users/49c00c60f91c4252809d831beb9b2496 => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:40.482322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb5c1c-46ad-41bc-b7f9-0c50d304249d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:40.535384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49c5d8-cb58-48e2-bb46-f5802e4c2171 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:40.535605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49c5d8-cb58-48e2-bb46-f5802e4c2171 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:40.535985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49c5d8-cb58-48e2-bb46-f5802e4c2171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:40.582731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3d49c5d8-cb58-48e2-bb46-f5802e4c2171 None None] MFA Rules not processed for user `770105b06e084179a819a54d28b70cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:40.587394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3d49c5d8-cb58-48e2-bb46-f5802e4c2171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:40.670604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:40.671452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:40.671642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:40.671920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:40.672569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:40.674555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:40.721839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:40.760050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 913/1786] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:40 2022] POST /identity/v3/users => generated 296 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:40.764224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:40.766317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:40.801184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 874/1787] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:40.812060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed41f4b-3f53-495c-86c1-790bcb5ca2e7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:40.836214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:40.836902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:40.837216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:40.837502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:40.845827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:40.852495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:40.899442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49c5d8-cb58-48e2-bb46-f5802e4c2171 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:40.900026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 914/1788] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:40.916388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49c5d8-cb58-48e2-bb46-f5802e4c2171 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:40.930172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:40.932039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:40.932433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:40.932822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] PATH_INFO: `/v3/projects/360520758fb040259145af09673e7c0c/users/c8e5deecb8534a56959a80472d7209e0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:40.972240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] RBAC: Authorizing `identity:create_grant(project_id=360520758fb040259145af09673e7c0c, user_id=c8e5deecb8534a56959a80472d7209e0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:40.981828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:41.007097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 875/1789] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:40 2022] PUT /identity/v3/projects/360520758fb040259145af09673e7c0c/users/c8e5deecb8534a56959a80472d7209e0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:41.011465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:41.035445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13064e19-ed65-4cd5-89cb-c9896c240eb1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:41.222106 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de590348-16af-43ad-9464-91c4b3b13192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:41.222956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de590348-16af-43ad-9464-91c4b3b13192 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:41.224108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de590348-16af-43ad-9464-91c4b3b13192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:41.224398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de590348-16af-43ad-9464-91c4b3b13192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:41.225449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de590348-16af-43ad-9464-91c4b3b13192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:41.229983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de590348-16af-43ad-9464-91c4b3b13192 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:41.237065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:41.262045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:41.270938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 915/1790] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:41.281016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ace2d-f359-4f88-93c0-5d4d6f76007c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:41.295936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:41.296913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:41.297336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:41.297681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] PATH_INFO: `/v3/projects/360520758fb040259145af09673e7c0c/users/c8e5deecb8534a56959a80472d7209e0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:41.307198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=360520758fb040259145af09673e7c0c, user_id=c8e5deecb8534a56959a80472d7209e0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:41.310257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:41.335407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 876/1791] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:41 2022] PUT /identity/v3/projects/360520758fb040259145af09673e7c0c/users/c8e5deecb8534a56959a80472d7209e0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:41.352336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:41.352779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:41.353072 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:41.444604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] MFA Rules not processed for user `c8e5deecb8534a56959a80472d7209e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:41.446216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:41.595499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 916/1792] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:41.610205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:41.613259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:41.692124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446360a9-bad9-4f19-ab98-01fb55de4d62 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:41.697728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:41.698613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:41.699074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:41.699635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:41.711177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:41.713009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:41.736754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 877/1793] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:42.164425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:42.165984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:42.289226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54393321-f8ef-4bea-b72f-8b5d6fe0bb9a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:42.295809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:42.296489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:42.296773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:42.297049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] PATH_INFO: `/v3/users/d09f5aee112041da83b745df8c4d536a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:42.299184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d09f5aee112041da83b745df8c4d536a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:42.301834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:42.363877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:42.379778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc89d4-8621-4025-93f9-6387745484a7 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:42.387810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e5deecb8534a56959a80472d7209e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '360520758fb040259145af09673e7c0c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:42.388969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:42.389624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:42.390258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:42.391535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:42.392095 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:42.394448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:42.398892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd09f5aee112041da83b745df8c4d536a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.414965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 878/1794] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:57:42.415182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd09f5aee112041da83b745df8c4d536a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.419242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:42.438167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac2b03-f0bb-467f-a577-5f2270d4587d tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:42.439633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd09f5aee112041da83b745df8c4d536a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.440488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd09f5aee112041da83b745df8c4d536a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.444833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e5deecb8534a56959a80472d7209e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '360520758fb040259145af09673e7c0c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:42.444833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:42.445422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:42.446255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:42.446801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:42.451470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:42.461889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-869b4fe8-8de5-46f2-8068-ad7ab4aa2df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd09f5aee112041da83b745df8c4d536a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.464698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:42.471514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 917/1795] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:42 2022] DELETE /identity/v3/users/d09f5aee112041da83b745df8c4d536a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:42.475645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:42.476059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:42.476703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:42.504497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 879/1796] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:42 2022] POST /identity/v3/projects => generated 352 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:57:42.509000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:42.512383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:42.530622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:42.536995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:42.694091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f7d4d-3877-4a8d-9904-96e03a1ab61e tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:42.701551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e5deecb8534a56959a80472d7209e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '360520758fb040259145af09673e7c0c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:42.702433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:42.702948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:42.703513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:42.704335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:42.707757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 918/1797] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:42.707757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:42.712463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:42.714445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:42.715410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:42.742825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 880/1798] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:42.753473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71904c8f-901d-46da-8b4b-d2d457b62cfb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:42.773409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:42.778320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:42.779268 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:42.779632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] PATH_INFO: `/v3/projects/debe98ba25a24934be06d95e854157f9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:42.782753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=debe98ba25a24934be06d95e854157f9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:42.785274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:42.813715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:42.825759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3189ee-563b-4ae1-bc52-a764dd9eaac8 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:42.831555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e5deecb8534a56959a80472d7209e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '360520758fb040259145af09673e7c0c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:42.832248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:42.832594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:42.833466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] PATH_INFO: `/v3/projects/53df21b9bac14634a6e01137b0c0bc23` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:42.834407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorizing `identity:delete_project(project_id=53df21b9bac14634a6e01137b0c0bc23)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:42.836843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:42.838989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:42.879992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'debe98ba25a24934be06d95e854157f9', 'request_id': 'req-b24bd4f5-4713-47b5-8882-0b3285133502'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.892699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'debe98ba25a24934be06d95e854157f9', 'request_id': 'req-b24bd4f5-4713-47b5-8882-0b3285133502'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.915737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'debe98ba25a24934be06d95e854157f9', 'request_id': 'req-b24bd4f5-4713-47b5-8882-0b3285133502'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.938623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] The token cache is being invalidate because project debe98ba25a24934be06d95e854157f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:42.939074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.939978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.941768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 919/1799] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:42 2022] DELETE /identity/v3/projects/debe98ba25a24934be06d95e854157f9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:42.945903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:42.947161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53df21b9bac14634a6e01137b0c0bc23', 'request_id': 'req-be0e2c2f-2011-4df8-a481-ddfd54aea051'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.949853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:42.965354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53df21b9bac14634a6e01137b0c0bc23', 'request_id': 'req-be0e2c2f-2011-4df8-a481-ddfd54aea051'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:42.984122 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53df21b9bac14634a6e01137b0c0bc23', 'request_id': 'req-be0e2c2f-2011-4df8-a481-ddfd54aea051'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.013888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] The token cache is being invalidate because project 53df21b9bac14634a6e01137b0c0bc23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:43.013888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.013888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be0e2c2f-2011-4df8-a481-ddfd54aea051 tempest-VolumeQuotaClassesTest-589484165 tempest-VolumeQuotaClassesTest-589484165-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.019035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 881/1800] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:42 2022] DELETE /identity/v3/projects/53df21b9bac14634a6e01137b0c0bc23 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:43.020568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacad635-03ad-477f-83bb-75259f329ced None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.020796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacad635-03ad-477f-83bb-75259f329ced None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.021562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacad635-03ad-477f-83bb-75259f329ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.097438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cacad635-03ad-477f-83bb-75259f329ced None None] MFA Rules not processed for user `68ef81de9b664b31989ba7e881013cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:43.100889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cacad635-03ad-477f-83bb-75259f329ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:43.104263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd4f5-4713-47b5-8882-0b3285133502 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:43.210967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:43.259237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:43.260716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.261060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.261593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.265546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:43.346559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 882/1801] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:43.351227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10510d-1f5d-4e2a-8a66-8f8de4a05fb8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.352845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10510d-1f5d-4e2a-8a66-8f8de4a05fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.353766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10510d-1f5d-4e2a-8a66-8f8de4a05fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.427603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:43.429052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:43.442313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1b10510d-1f5d-4e2a-8a66-8f8de4a05fb8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:43.443538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1b10510d-1f5d-4e2a-8a66-8f8de4a05fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:43.454584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 920/1802] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:43.455979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:43.466835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:43.470544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 883/1803] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:43.478809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ce78-218a-4ff6-8911-99a5549d7cec None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.479391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ce78-218a-4ff6-8911-99a5549d7cec None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.479836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ce78-218a-4ff6-8911-99a5549d7cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.490578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc372df5-89ef-47cd-8a49-e5f996d3e576 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:43.508496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:43.509373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.509652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.509922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] PATH_INFO: `/v3/users/6bdfc0fab3d34699ab16a78cbabff844` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.511702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bdfc0fab3d34699ab16a78cbabff844)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:43.513683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:43.535320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5042ce78-218a-4ff6-8911-99a5549d7cec None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:43.546940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5042ce78-218a-4ff6-8911-99a5549d7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:43.566065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 884/1804] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:57:43.570439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ce78-218a-4ff6-8911-99a5549d7cec None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:43.588014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ce78-218a-4ff6-8911-99a5549d7cec None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:43.599025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:43.600471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.600934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.601457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.603490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:43.604304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:43.606845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:43.627211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 885/1805] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:57:43.631832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:43.650504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bdfc0fab3d34699ab16a78cbabff844'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.658238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f27b4-e1b9-4911-8cdc-7ca1d210bcc3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:43.662143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bdfc0fab3d34699ab16a78cbabff844'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.672322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa7067-8bde-47ec-a8fa-bd81a8c6ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:43.673775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa7067-8bde-47ec-a8fa-bd81a8c6ca79 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.674229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa7067-8bde-47ec-a8fa-bd81a8c6ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.674653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa7067-8bde-47ec-a8fa-bd81a8c6ca79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.675661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa7067-8bde-47ec-a8fa-bd81a8c6ca79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:43.681486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa7067-8bde-47ec-a8fa-bd81a8c6ca79 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:43.711443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bdfc0fab3d34699ab16a78cbabff844'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.713495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bdfc0fab3d34699ab16a78cbabff844'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.732007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bdfc0fab3d34699ab16a78cbabff844'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:43.759349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 886/1806] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:43 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:57:43.759656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 921/1807] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:43 2022] DELETE /identity/v3/users/6bdfc0fab3d34699ab16a78cbabff844 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:43.766983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:43.767274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:43.767555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:43.770390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:43.772911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:43.835597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:43.836134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:43.907086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:43.972478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289afa09-59e5-4867-a750-2605ffd1b8db admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:43.986444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:44.015171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 887/1808] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:44.102911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.103553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.103863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.104094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.104739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.106965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.135042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 922/1809] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:43 2022] POST /identity/v3/users => generated 292 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:44.135273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.137336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:44.140397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.161128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a9f9e-ae94-4d69-b55f-c649f9c7b574 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.178379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.178663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.178921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.179195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.179854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.181999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.205043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 923/1810] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:44.210491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.228514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8f431-7045-4b22-81fb-514d955d5c8f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.243144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.243784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.244070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.244350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.245002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.245109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd3621-4828-4cdc-93a4-b2e1b14e32f0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.247203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.252394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32633d03-8921-4c6c-ba90-29cee8e364b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.255462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32633d03-8921-4c6c-ba90-29cee8e364b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.255851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32633d03-8921-4c6c-ba90-29cee8e364b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.256139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32633d03-8921-4c6c-ba90-29cee8e364b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.270755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 924/1811] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:57:44.276170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.281300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32633d03-8921-4c6c-ba90-29cee8e364b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.283217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32633d03-8921-4c6c-ba90-29cee8e364b2 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.295041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6de0c9-61a2-404e-8469-b791afef57ae admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.303315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 888/1812] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:57:44.305601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.306705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.307211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.307653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] PATH_INFO: `/v3/projects/8db3a6e7dcbf43ffa9d467dce1dadb1b/users/e3e80221bb5a46dea277b2787e21cfc5/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.329772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db3a6e7dcbf43ffa9d467dce1dadb1b, user_id=e3e80221bb5a46dea277b2787e21cfc5, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.337389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.359028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 925/1813] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:44 2022] PUT /identity/v3/projects/8db3a6e7dcbf43ffa9d467dce1dadb1b/users/e3e80221bb5a46dea277b2787e21cfc5/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:44.377730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9573c-8673-4ec4-8098-dabce5ed9ed7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.377730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9573c-8673-4ec4-8098-dabce5ed9ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.378161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9573c-8673-4ec4-8098-dabce5ed9ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.419892 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-25d9573c-8673-4ec4-8098-dabce5ed9ed7 None None] MFA Rules not processed for user `e3e80221bb5a46dea277b2787e21cfc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:44.422015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-25d9573c-8673-4ec4-8098-dabce5ed9ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:44.520718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 889/1814] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:44.522344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.613279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fb448-d710-4918-a25f-ad077ff1f40e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.623181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.623936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.625016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.626157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.629607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.634023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.685863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 926/1815] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:44 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:57:44.687295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9573c-8673-4ec4-8098-dabce5ed9ed7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.702275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9573c-8673-4ec4-8098-dabce5ed9ed7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.808962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.809788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.810207 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.810612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.812012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.814632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.831376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:44.860437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 890/1816] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:44 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:44.862440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.877569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7ffbc-5845-4e4f-90cc-653f07b8c0ad admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.883266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.884248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.884715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.885092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:44.893782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:44.896635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:44.913398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 927/1817] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:44.921571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:44.956462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d23e9a-dc7a-45e6-980c-0e3dc57b5524 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:44.971802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:44.972707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:44.973026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:44.973925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] PATH_INFO: `/v3/projects/a8d76d6d8be047f0acb4b9a13c9efde2/users/8c8689ae7a2e4fd294a993a4d1171fc0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.001661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d76d6d8be047f0acb4b9a13c9efde2, user_id=8c8689ae7a2e4fd294a993a4d1171fc0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.004396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.034657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 891/1818] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:44 2022] PUT /identity/v3/projects/a8d76d6d8be047f0acb4b9a13c9efde2/users/8c8689ae7a2e4fd294a993a4d1171fc0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:45.035676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.047620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb6135-00d0-4b50-b4b9-1d4d65c6a374 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:45.081100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:45.140575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:45.141092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.141397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.141669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.142309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.144046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.153869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.155672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.174458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 928/1819] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:57:45.174598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f6e3-5c6d-44a8-b042-039ffa56617b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:45.176069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.187671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:45.188250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.189051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.189051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] PATH_INFO: `/v3/projects/339ff606136346b7a5c81c533ef101b9` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.190046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=339ff606136346b7a5c81c533ef101b9)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.191935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.196529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b3da7-fc9c-44f9-82fb-9a88302a989f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:45.225833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5fb8e-27a0-48f0-a48f-bfc8933ee339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:45.226547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fb8e-27a0-48f0-a48f-bfc8933ee339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.227227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fb8e-27a0-48f0-a48f-bfc8933ee339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.227227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fb8e-27a0-48f0-a48f-bfc8933ee339 admin admin] PATH_INFO: `/v3/projects/a8d76d6d8be047f0acb4b9a13c9efde2/users/8c8689ae7a2e4fd294a993a4d1171fc0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.232627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5fb8e-27a0-48f0-a48f-bfc8933ee339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d76d6d8be047f0acb4b9a13c9efde2, user_id=8c8689ae7a2e4fd294a993a4d1171fc0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.243667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5fb8e-27a0-48f0-a48f-bfc8933ee339 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.281396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 929/1820] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:45 2022] PUT /identity/v3/projects/a8d76d6d8be047f0acb4b9a13c9efde2/users/8c8689ae7a2e4fd294a993a4d1171fc0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:45.293902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.294159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.294458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.344169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] MFA Rules not processed for user `8c8689ae7a2e4fd294a993a4d1171fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:45.345655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.347051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '339ff606136346b7a5c81c533ef101b9', 'request_id': 'req-4a108092-0b2f-4f6b-8bc1-68290446a1ac'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.354803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '339ff606136346b7a5c81c533ef101b9', 'request_id': 'req-4a108092-0b2f-4f6b-8bc1-68290446a1ac'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.365620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '339ff606136346b7a5c81c533ef101b9', 'request_id': 'req-4a108092-0b2f-4f6b-8bc1-68290446a1ac'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.384309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] The token cache is being invalidate because project 339ff606136346b7a5c81c533ef101b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:45.384598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.386877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.390971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 892/1821] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:45 2022] DELETE /identity/v3/projects/339ff606136346b7a5c81c533ef101b9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:45.392173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.394041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.424156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 930/1822] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:45.437449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.439211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.481710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a108092-0b2f-4f6b-8bc1-68290446a1ac admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:45.488829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:45.489817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.490258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.490533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] PATH_INFO: `/v3/users/d482d07bf6ca40b89ed40348817c8e84` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.491783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d482d07bf6ca40b89ed40348817c8e84)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.493929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.502402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20fd48-907a-40bb-90d5-788ec4adf209 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:45.506309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b4a5d-6066-4de2-8a78-474435cde87c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:45.506936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4a5d-6066-4de2-8a78-474435cde87c service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.507216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4a5d-6066-4de2-8a78-474435cde87c service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.507490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4a5d-6066-4de2-8a78-474435cde87c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.519147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b4a5d-6066-4de2-8a78-474435cde87c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.520667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b4a5d-6066-4de2-8a78-474435cde87c service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.535967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 931/1823] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:57:45.565284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd482d07bf6ca40b89ed40348817c8e84'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.568683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=5,keystone:SELECT=81,keystone:DELETE=14 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:45.586710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd482d07bf6ca40b89ed40348817c8e84'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.595574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd482d07bf6ca40b89ed40348817c8e84'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.614583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd482d07bf6ca40b89ed40348817c8e84'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.615543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd482d07bf6ca40b89ed40348817c8e84'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.625253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 893/1824] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:45 2022] DELETE /identity/v3/users/d482d07bf6ca40b89ed40348817c8e84 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:45.658063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a858-3768-459c-9098-84c02c15cda2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.658580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.659574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a858-3768-459c-9098-84c02c15cda2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.660404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.660754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63a858-3768-459c-9098-84c02c15cda2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.666414 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 932/1825] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:57:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:45.667446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63a858-3768-459c-9098-84c02c15cda2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.669928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2d63a858-3768-459c-9098-84c02c15cda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.723430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:45.769080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c902e-503b-4705-8c0b-045930be269d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:45.773413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63a858-3768-459c-9098-84c02c15cda2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:45.774724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742888ce-b48c-4e04-b602-09fbd572d021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:45.775693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742888ce-b48c-4e04-b602-09fbd572d021 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.776041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742888ce-b48c-4e04-b602-09fbd572d021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.776394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742888ce-b48c-4e04-b602-09fbd572d021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.779209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-742888ce-b48c-4e04-b602-09fbd572d021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.779646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:45.780359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.780700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.781080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] PATH_INFO: `/v3/projects/f03b52573d42491eb46c4a4315a9fc3b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.782768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03b52573d42491eb46c4a4315a9fc3b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.785305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.868750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03b52573d42491eb46c4a4315a9fc3b', 'request_id': 'req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.894475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03b52573d42491eb46c4a4315a9fc3b', 'request_id': 'req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.908007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03b52573d42491eb46c4a4315a9fc3b', 'request_id': 'req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.910271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742888ce-b48c-4e04-b602-09fbd572d021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:45.916224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742888ce-b48c-4e04-b602-09fbd572d021 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:45.935998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] The token cache is being invalidate because project f03b52573d42491eb46c4a4315a9fc3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:45.936287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.937230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:45.939284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 894/1826] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:57:45.940935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:45.941662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:45.941951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:45.951849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 933/1827] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:45 2022] DELETE /identity/v3/projects/f03b52573d42491eb46c4a4315a9fc3b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:45.958104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:45.964373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:45.979512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:45.982745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:46.164346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5ca6c-8703-4678-97fc-ec3348dbda9a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:46.195710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:46.196368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:46.196649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:46.196937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] PATH_INFO: `/v3/users/c8e5deecb8534a56959a80472d7209e0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:46.200907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e5deecb8534a56959a80472d7209e0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:46.203590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:46.252465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 895/1828] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:46.310503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:46.312271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:46.324822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e5deecb8534a56959a80472d7209e0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.333871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e5deecb8534a56959a80472d7209e0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.353231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62125-f5cc-43d5-9be4-fc05e013a48c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:46.353231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7912a1-c83b-4b56-bffb-dedbf817273a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:46.353231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7912a1-c83b-4b56-bffb-dedbf817273a service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:46.353231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7912a1-c83b-4b56-bffb-dedbf817273a service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:46.354027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7912a1-c83b-4b56-bffb-dedbf817273a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:46.354124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:46.354614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e5deecb8534a56959a80472d7209e0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.355447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e5deecb8534a56959a80472d7209e0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.378267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e5deecb8534a56959a80472d7209e0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.387247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7912a1-c83b-4b56-bffb-dedbf817273a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:46.390632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7912a1-c83b-4b56-bffb-dedbf817273a service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:46.394198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 934/1829] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:45 2022] DELETE /identity/v3/users/c8e5deecb8534a56959a80472d7209e0 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:46.423638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 896/1830] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:46.476840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:46.478448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:46.511015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb17af-3a9c-487a-a666-9adcf3657994 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:46.512343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb17af-3a9c-487a-a666-9adcf3657994 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:46.512623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb17af-3a9c-487a-a666-9adcf3657994 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:46.514037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 897/1831] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 14:57:46.550384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1090b-697c-4317-9b5e-c69fd9b587c0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:46.551363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1090b-697c-4317-9b5e-c69fd9b587c0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:46.551847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1090b-697c-4317-9b5e-c69fd9b587c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:46.556273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3125d5-7a7e-4c7d-90ad-aff6c7832c66 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:46.561908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:46.562552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:46.562887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:46.563164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] PATH_INFO: `/v3/users/e3e80221bb5a46dea277b2787e21cfc5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:46.564332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e80221bb5a46dea277b2787e21cfc5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:46.565016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:46.566324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:46.613392 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c8a1090b-697c-4317-9b5e-c69fd9b587c0 None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:46.622872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c8a1090b-697c-4317-9b5e-c69fd9b587c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:46.787083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e80221bb5a46dea277b2787e21cfc5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.805916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e80221bb5a46dea277b2787e21cfc5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.847279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 898/1832] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:46.854573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e80221bb5a46dea277b2787e21cfc5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.856686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e80221bb5a46dea277b2787e21cfc5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.870280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e80221bb5a46dea277b2787e21cfc5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:46.877600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 935/1833] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:46 2022] DELETE /identity/v3/users/e3e80221bb5a46dea277b2787e21cfc5 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:46.882219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:46.882503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:46.882779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:46.904963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:46.906417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:46.937107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:46.938818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:47.037940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 899/1834] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:47.061158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:47.063741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:47.094265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:47.109914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25357a84-c31d-430e-b02b-b19e3781d8e7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:47.120500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:47.121774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:47.122061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:47.122374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:47.126048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:47.143397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:47.146113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:47.163717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 936/1835] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:47.187535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3b46c-7b74-48ad-83f7-31c9cd937f82 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:47.191836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:47.192459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:47.192739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:47.193015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:47.203678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:47.205127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:47.217419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 900/1836] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:47.856012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:47.859225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:47.877470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cfea5-875f-4d38-a04c-46973987e3c9 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:47.898518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:47.899269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:47.899673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:47.901598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] PATH_INFO: `/v3/users/879a7af9d8384b469ad8b684066d0ec9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:47.903932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=879a7af9d8384b469ad8b684066d0ec9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:47.907261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:47.981718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '879a7af9d8384b469ad8b684066d0ec9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:47.993663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '879a7af9d8384b469ad8b684066d0ec9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.011950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '879a7af9d8384b469ad8b684066d0ec9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.021635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '879a7af9d8384b469ad8b684066d0ec9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.034614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9cd2242-4954-435b-988a-8aea3ef7c2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '879a7af9d8384b469ad8b684066d0ec9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.042799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 937/1837] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:47 2022] DELETE /identity/v3/users/879a7af9d8384b469ad8b684066d0ec9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:48.064040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:48.065593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:48.086032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e631a-9991-4375-a4f5-ff2e25fffc80 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:48.172652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:48.193316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:48.194063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.194341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.194607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] PATH_INFO: `/v3/projects/f0af9ab255c14c22b887f271c8faf173` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.195787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0af9ab255c14c22b887f271c8faf173)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:48.197904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:48.256813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0af9ab255c14c22b887f271c8faf173', 'request_id': 'req-d234f993-2998-4929-9e1d-256ba68f28fb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.264681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0af9ab255c14c22b887f271c8faf173', 'request_id': 'req-d234f993-2998-4929-9e1d-256ba68f28fb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.271199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0af9ab255c14c22b887f271c8faf173', 'request_id': 'req-d234f993-2998-4929-9e1d-256ba68f28fb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.289435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] The token cache is being invalidate because project f0af9ab255c14c22b887f271c8faf173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:48.289761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.290725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d234f993-2998-4929-9e1d-256ba68f28fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.295575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 901/1838] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:48 2022] DELETE /identity/v3/projects/f0af9ab255c14c22b887f271c8faf173 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:48.297263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6e58c-21bd-45e2-a69d-e187aa6db26c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.297567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6e58c-21bd-45e2-a69d-e187aa6db26c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.297845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6e58c-21bd-45e2-a69d-e187aa6db26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.318154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fef6e58c-21bd-45e2-a69d-e187aa6db26c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:48.319511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fef6e58c-21bd-45e2-a69d-e187aa6db26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:48.394919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 938/1839] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:48.403345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313472b-8c6c-4110-b5ec-b6a59d41121b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.403630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313472b-8c6c-4110-b5ec-b6a59d41121b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.403899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313472b-8c6c-4110-b5ec-b6a59d41121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.435785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8313472b-8c6c-4110-b5ec-b6a59d41121b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:48.437049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8313472b-8c6c-4110-b5ec-b6a59d41121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:48.450259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 902/1840] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:48.451479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6e58c-21bd-45e2-a69d-e187aa6db26c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:48.461849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6e58c-21bd-45e2-a69d-e187aa6db26c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:48.478986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:48.480001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.480676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.481558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.482444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:48.483206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:48.485400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:48.508079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 939/1841] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:48.510942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313472b-8c6c-4110-b5ec-b6a59d41121b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:48.530537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313472b-8c6c-4110-b5ec-b6a59d41121b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:48.535522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:48.538198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:48.538813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.539144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.539428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.539940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:48.540094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:48.542505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:48.562590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893920c3-2a0b-4748-99a5-666c6c19ce54 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:48.577263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:48.580024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.580320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.581365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] PATH_INFO: `/v3/projects/7b907e5fa35a49ad9ba11718211a3275` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.584251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b907e5fa35a49ad9ba11718211a3275)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:48.585781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 903/1842] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:48 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:48.590764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:48.590881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:48.689967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:48.712862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b907e5fa35a49ad9ba11718211a3275', 'request_id': 'req-e07fa761-52e8-4109-9f24-97e356bf791c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.722969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca360b-42bf-426e-8fc2-ff5adc7e743d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:48.758931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b907e5fa35a49ad9ba11718211a3275', 'request_id': 'req-e07fa761-52e8-4109-9f24-97e356bf791c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.770370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b907e5fa35a49ad9ba11718211a3275', 'request_id': 'req-e07fa761-52e8-4109-9f24-97e356bf791c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.800229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] The token cache is being invalidate because project 7b907e5fa35a49ad9ba11718211a3275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:48.800589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.801753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:48.807838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 940/1843] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:48 2022] DELETE /identity/v3/projects/7b907e5fa35a49ad9ba11718211a3275 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:48.810009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:48.813505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:48.846261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:48.869312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:48.869942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.870280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.871317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.872010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:48.875430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:48.939057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 904/1844] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:48 2022] POST /identity/v3/users => generated 287 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:48.940088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:48.947126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:48.968760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fa761-52e8-4109-9f24-97e356bf791c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:48.971395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36856-060d-4fde-8739-3df2977e7c8e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:48.984992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:48.985709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.986008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.986270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] PATH_INFO: `/v3/users/68ef81de9b664b31989ba7e881013cfa` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.987093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:48.987860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:48.988208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:48.988548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] PATH_INFO: `/v3/projects/8db3a6e7dcbf43ffa9d467dce1dadb1b` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:48.989849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db3a6e7dcbf43ffa9d467dce1dadb1b)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:48.990791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ef81de9b664b31989ba7e881013cfa)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:48.992667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:48.992791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:49.088689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ef81de9b664b31989ba7e881013cfa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.098089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ef81de9b664b31989ba7e881013cfa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.115437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ef81de9b664b31989ba7e881013cfa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.118327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ef81de9b664b31989ba7e881013cfa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.119553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db3a6e7dcbf43ffa9d467dce1dadb1b', 'request_id': 'req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.133528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ef81de9b664b31989ba7e881013cfa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.144388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 941/1845] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:48 2022] DELETE /identity/v3/users/68ef81de9b664b31989ba7e881013cfa => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:49.145826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:49.150685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db3a6e7dcbf43ffa9d467dce1dadb1b', 'request_id': 'req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.154903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.170440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db3a6e7dcbf43ffa9d467dce1dadb1b', 'request_id': 'req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.196171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] The token cache is being invalidate because project 8db3a6e7dcbf43ffa9d467dce1dadb1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:49.196610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.198315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.203517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 905/1846] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:48 2022] DELETE /identity/v3/projects/8db3a6e7dcbf43ffa9d467dce1dadb1b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:49.203517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:49.207562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.259957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:49.282664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c8b20-6343-441c-9cb0-649e71326511 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:49.294707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:49.295366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.295643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.295944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] PATH_INFO: `/v3/projects/360520758fb040259145af09673e7c0c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.297047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=360520758fb040259145af09673e7c0c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:49.299002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:49.328918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d43763-a3a2-41fb-bc07-f6f31fb9e13e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:49.338870 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:49.345462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.345562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.345847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.346660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:49.348617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:49.377020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 906/1847] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:49.378026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:49.381988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.404286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360520758fb040259145af09673e7c0c', 'request_id': 'req-2b48e3cb-eae0-416a-b637-edf99d67ec60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.412845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a02e3d-8793-4d43-8605-093dd9509249 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:49.425164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360520758fb040259145af09673e7c0c', 'request_id': 'req-2b48e3cb-eae0-416a-b637-edf99d67ec60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.432375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360520758fb040259145af09673e7c0c', 'request_id': 'req-2b48e3cb-eae0-416a-b637-edf99d67ec60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.454467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] The token cache is being invalidate because project 360520758fb040259145af09673e7c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:49.454809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.455893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2b48e3cb-eae0-416a-b637-edf99d67ec60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.463057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 942/1848] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:49 2022] DELETE /identity/v3/projects/360520758fb040259145af09673e7c0c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:49.466791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d6f89-cdcc-4a49-9f6d-7c80755ed739 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.466791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d6f89-cdcc-4a49-9f6d-7c80755ed739 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.466791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d6f89-cdcc-4a49-9f6d-7c80755ed739 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.469020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 943/1849] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:57:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:57:49.469844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d6f89-cdcc-4a49-9f6d-7c80755ed739 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:49.472040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-835d6f89-cdcc-4a49-9f6d-7c80755ed739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.547179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:49.565040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:49.565756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.567363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.567363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] PATH_INFO: `/v3/users/8c8689ae7a2e4fd294a993a4d1171fc0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.570703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c8689ae7a2e4fd294a993a4d1171fc0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:49.576344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:49.620103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d6f89-cdcc-4a49-9f6d-7c80755ed739 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:49.626850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07413fea-52bc-4a3b-b5c1-3827303b8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:49.628019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07413fea-52bc-4a3b-b5c1-3827303b8829 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.628481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07413fea-52bc-4a3b-b5c1-3827303b8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.628760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07413fea-52bc-4a3b-b5c1-3827303b8829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.629454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07413fea-52bc-4a3b-b5c1-3827303b8829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:49.631113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07413fea-52bc-4a3b-b5c1-3827303b8829 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:49.650031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 944/1850] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:49.652699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b539bb-014f-4680-afc1-0506212b8d5e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.653462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b539bb-014f-4680-afc1-0506212b8d5e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.653946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b539bb-014f-4680-afc1-0506212b8d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.678270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c8689ae7a2e4fd294a993a4d1171fc0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.693904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d2b539bb-014f-4680-afc1-0506212b8d5e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:49.695160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d2b539bb-014f-4680-afc1-0506212b8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.718357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c8689ae7a2e4fd294a993a4d1171fc0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.728363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 945/1851] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:49.731162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.731795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.732284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.736405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c8689ae7a2e4fd294a993a4d1171fc0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.752276 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c8689ae7a2e4fd294a993a4d1171fc0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.753247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c8689ae7a2e4fd294a993a4d1171fc0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.781745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 907/1852] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:49 2022] DELETE /identity/v3/users/8c8689ae7a2e4fd294a993a4d1171fc0 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:49.785697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:49.788562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.809623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:49.810901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.888374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02da23-7209-444a-91d7-d2e76441b39f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:49.897739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:49.898386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:49.898665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:49.898936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] PATH_INFO: `/v3/projects/02743e3f4fb54c14b3e27331dbbdb9ae` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:49.900865 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02743e3f4fb54c14b3e27331dbbdb9ae)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:49.902861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:49.935313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 946/1853] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:57:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:49.936774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:49.938678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:49.951077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:49.956980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7152f0-b0e5-4fb4-9dd6-12aafbe6c60d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:49.971554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02743e3f4fb54c14b3e27331dbbdb9ae', 'request_id': 'req-df7c8443-d414-4278-b805-123042b531b6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.980905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02743e3f4fb54c14b3e27331dbbdb9ae', 'request_id': 'req-df7c8443-d414-4278-b805-123042b531b6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:49.990171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02743e3f4fb54c14b3e27331dbbdb9ae', 'request_id': 'req-df7c8443-d414-4278-b805-123042b531b6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.009877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] The token cache is being invalidate because project 02743e3f4fb54c14b3e27331dbbdb9ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:50.010626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.013246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-df7c8443-d414-4278-b805-123042b531b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.015123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 908/1854] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:49 2022] DELETE /identity/v3/projects/02743e3f4fb54c14b3e27331dbbdb9ae => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:50.017250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed75963-06b3-406c-88fd-0afb0cbe1a86 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.017802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed75963-06b3-406c-88fd-0afb0cbe1a86 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.018508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed75963-06b3-406c-88fd-0afb0cbe1a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.039493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.040257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.040598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.040806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] PATH_INFO: `/v3/projects/bd3cfdafe89243748fe29bef991b2215/users/ca360b149ac8402282733edfc4c67f9e/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.045044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:50.054366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7ed75963-06b3-406c-88fd-0afb0cbe1a86 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:50.061185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7ed75963-06b3-406c-88fd-0afb0cbe1a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.078309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 909/1855] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:50.078309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0cf4-175e-48f1-aa00-b26dce1016f0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.078309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0cf4-175e-48f1-aa00-b26dce1016f0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.078309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0cf4-175e-48f1-aa00-b26dce1016f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.082551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd3cfdafe89243748fe29bef991b2215, user_id=ca360b149ac8402282733edfc4c67f9e, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.087358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.112032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 947/1856] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:49 2022] PUT /identity/v3/projects/bd3cfdafe89243748fe29bef991b2215/users/ca360b149ac8402282733edfc4c67f9e/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:50.113127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.118617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-880b0cf4-175e-48f1-aa00-b26dce1016f0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:50.128919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-880b0cf4-175e-48f1-aa00-b26dce1016f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.214024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244abe1-ec3a-40c2-8191-bbc25f4d12ae admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.220616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab463a06-6615-4f76-8a94-a66d2b6a2b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.221521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463a06-6615-4f76-8a94-a66d2b6a2b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.221928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463a06-6615-4f76-8a94-a66d2b6a2b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.222651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463a06-6615-4f76-8a94-a66d2b6a2b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.223325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463a06-6615-4f76-8a94-a66d2b6a2b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:50.223622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463a06-6615-4f76-8a94-a66d2b6a2b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.225350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463a06-6615-4f76-8a94-a66d2b6a2b1d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.239218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 948/1857] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:50.240617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7372-55ea-4755-a3be-f3eed450b32b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.240911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7372-55ea-4755-a3be-f3eed450b32b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.241171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7372-55ea-4755-a3be-f3eed450b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.257862 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 910/1858] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:50.265463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b0cf4-175e-48f1-aa00-b26dce1016f0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.268212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-670b7372-55ea-4755-a3be-f3eed450b32b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:50.271757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-670b7372-55ea-4755-a3be-f3eed450b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.275714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b0cf4-175e-48f1-aa00-b26dce1016f0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.281153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b38b-7c7f-4f38-9cc4-1f70a7367e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.281900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b38b-7c7f-4f38-9cc4-1f70a7367e2a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.282245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b38b-7c7f-4f38-9cc4-1f70a7367e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.282572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b38b-7c7f-4f38-9cc4-1f70a7367e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.283280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21b38b-7c7f-4f38-9cc4-1f70a7367e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.285370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21b38b-7c7f-4f38-9cc4-1f70a7367e2a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.320409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 911/1859] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:50 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:50.321744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.322172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.323048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.325560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 949/1860] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:50.327818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7372-55ea-4755-a3be-f3eed450b32b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.340767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7372-55ea-4755-a3be-f3eed450b32b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.349756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:50.351319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.443660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.447173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.447173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.447173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.447173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.447973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.462578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 912/1861] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:50.464848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.469564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.478898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 950/1862] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:50 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:50.480872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.483043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.490985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:50.576973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de3d85-fe65-4657-9641-2ffed41f9591 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.582254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.582996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.583345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.584083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.600266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.602629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.621047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 951/1863] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:50.621968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.623691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.637333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3947a-1935-4c18-b4eb-9b5e2720598e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.644019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.644793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.645086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.645395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.647317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:50.658762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140a7ba-0af4-4fb7-878a-ad43b43b5bc4 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.662559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.663939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.671868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.673685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.674287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.674894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] PATH_INFO: `/v3/projects/a8d76d6d8be047f0acb4b9a13c9efde2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.676318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 913/1864] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:50.677278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.678734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d76d6d8be047f0acb4b9a13c9efde2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.684596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:50.687442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.690045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d0ea-0d53-4d19-8aee-d30dad541041 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.696660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.697658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.698131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.698799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.699711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:50.700283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.702686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.742469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 914/1865] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:50.743669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.770242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3271a6-76c1-4e62-86df-f12df6f067dc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.784231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:50.784964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:50.785139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:50.785443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:50.788434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:50.794011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:50.831886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 915/1866] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:50.832838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d76d6d8be047f0acb4b9a13c9efde2', 'request_id': 'req-03e4dd7c-404f-4207-95ba-84d67e240d60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.836515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.852418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19435d0b-6999-4bd1-935f-bd4c2346f943 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:50.860440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d76d6d8be047f0acb4b9a13c9efde2', 'request_id': 'req-03e4dd7c-404f-4207-95ba-84d67e240d60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.878609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d76d6d8be047f0acb4b9a13c9efde2', 'request_id': 'req-03e4dd7c-404f-4207-95ba-84d67e240d60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.907576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] The token cache is being invalidate because project a8d76d6d8be047f0acb4b9a13c9efde2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:50.910333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.912549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:50.918336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 952/1867] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:50 2022] DELETE /identity/v3/projects/a8d76d6d8be047f0acb4b9a13c9efde2 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:50.919451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:50.922786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:51.004208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeab28a-4d88-4d88-a83b-d3d6102de308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.005803 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab28a-4d88-4d88-a83b-d3d6102de308 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.007326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab28a-4d88-4d88-a83b-d3d6102de308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.007716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab28a-4d88-4d88-a83b-d3d6102de308 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.029008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeab28a-4d88-4d88-a83b-d3d6102de308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.031583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeab28a-4d88-4d88-a83b-d3d6102de308 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.104620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 916/1868] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:50 2022] POST /identity/v3/projects => generated 367 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:51.104620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.104620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.104620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.113109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4dd7c-404f-4207-95ba-84d67e240d60 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.129268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:51.130932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:51.164874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:51.211105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7350f86-2c80-47c3-ad1a-cdfe7123d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.212017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350f86-2c80-47c3-ad1a-cdfe7123d535 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.212562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350f86-2c80-47c3-ad1a-cdfe7123d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.213140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350f86-2c80-47c3-ad1a-cdfe7123d535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.214009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350f86-2c80-47c3-ad1a-cdfe7123d535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.216456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 917/1869] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:51.216644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350f86-2c80-47c3-ad1a-cdfe7123d535 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.217589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.219059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:51.233142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 953/1870] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:51.235028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebc9bb-dbd4-46ca-8d17-8bac87df0611 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.235492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebc9bb-dbd4-46ca-8d17-8bac87df0611 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.235955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebc9bb-dbd4-46ca-8d17-8bac87df0611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.259649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70903c68-dd49-4021-9530-05b3ce40aa59 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.264498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-59ebc9bb-dbd4-46ca-8d17-8bac87df0611 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:51.266421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-59ebc9bb-dbd4-46ca-8d17-8bac87df0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:51.279892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 954/1871] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:51.282082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebc9bb-dbd4-46ca-8d17-8bac87df0611 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.290164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.292028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.292485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.294767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.295631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.299099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebc9bb-dbd4-46ca-8d17-8bac87df0611 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.307386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.308500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.309711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.310237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] PATH_INFO: `/v3/projects/3eec7115e55640c3ba320071a7352a70/users/43baaf0e01ee4bc8b7ec5e94d6a743b5/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.315344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.336620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eec7115e55640c3ba320071a7352a70, user_id=43baaf0e01ee4bc8b7ec5e94d6a743b5, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.343828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.369042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 918/1872] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:51.378198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.385078 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 955/1873] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:51 2022] PUT /identity/v3/projects/3eec7115e55640c3ba320071a7352a70/users/43baaf0e01ee4bc8b7ec5e94d6a743b5/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:51.386794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.403068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd9b9-7aa4-401e-a05c-4031afa7a954 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.503983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:51.520955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f27278-7430-4c3c-bd69-2a411fd8f912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.522098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27278-7430-4c3c-bd69-2a411fd8f912 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.522748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27278-7430-4c3c-bd69-2a411fd8f912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.523613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27278-7430-4c3c-bd69-2a411fd8f912 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.524150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27278-7430-4c3c-bd69-2a411fd8f912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:57:51.524560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27278-7430-4c3c-bd69-2a411fd8f912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.526746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27278-7430-4c3c-bd69-2a411fd8f912 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.543797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 919/1874] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:57:51.543797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6121e-0845-4e3c-acbe-6121650e19ca None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.543797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6121e-0845-4e3c-acbe-6121650e19ca None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.543797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6121e-0845-4e3c-acbe-6121650e19ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.547562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487eadc-4a96-4846-ac9d-074bf1fb492a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.556767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.557738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.559270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.559559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.560220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.561914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.574809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 956/1875] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:51.575610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.590259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-edd6121e-0845-4e3c-acbe-6121650e19ca None None] MFA Rules not processed for user `43baaf0e01ee4bc8b7ec5e94d6a743b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:51.591787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-edd6121e-0845-4e3c-acbe-6121650e19ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:51.602068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f9a81-150a-4130-960f-c690fca12adf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.621911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.623571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.623972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.624360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.625128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.627539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.704816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 957/1876] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:51.705839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.724404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 920/1877] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:51.725069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6121e-0845-4e3c-acbe-6121650e19ca None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.743674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6121e-0845-4e3c-acbe-6121650e19ca None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.803637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73a5bf-1f0e-4e6e-b95e-4e92bc61a647 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.814683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.815855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.816326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.816652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.817356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.819106 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.836623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 958/1878] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:51.836623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.840229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.840873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.841446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.841745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.842417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.844643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.859706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3d235-8a5c-43e2-a989-ce095f69cebc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.875169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.875834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.876167 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.876450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.877667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.880242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.888261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 921/1879] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:51.889213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.906567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:51.919971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed38d8-0827-4db6-8d39-cf082485bb99 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:51.926329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:51.927333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:51.927859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:51.928517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] PATH_INFO: `/v3/projects/e70b29cddf6f44689ae6b05afbf67dcd/users/61af4255e31c4c7a8d523487f123e345/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:51.940129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 959/1880] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:51.943887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:51.952694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70b29cddf6f44689ae6b05afbf67dcd, user_id=61af4255e31c4c7a8d523487f123e345, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:51.955920 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:51.980642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 922/1881] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:51 2022] PUT /identity/v3/projects/e70b29cddf6f44689ae6b05afbf67dcd/users/61af4255e31c4c7a8d523487f123e345/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:51.981800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.051933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:52.075600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828b2c6-a522-4d30-b436-b7e96e3fae03 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.097409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f6fa2-216b-4cf0-b3ac-fe36bb83204b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.099217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7072b98-9719-4030-8654-74cf0fc6ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.099770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072b98-9719-4030-8654-74cf0fc6ab24 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.100131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072b98-9719-4030-8654-74cf0fc6ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.100479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072b98-9719-4030-8654-74cf0fc6ab24 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.101613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7072b98-9719-4030-8654-74cf0fc6ab24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.103866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7072b98-9719-4030-8654-74cf0fc6ab24 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.138828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 923/1882] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:51 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:52.140910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8e84-d517-4ca3-bf7d-f07d85890033 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.141472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8e84-d517-4ca3-bf7d-f07d85890033 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.142242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8e84-d517-4ca3-bf7d-f07d85890033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.159271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.160395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.160656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.161313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.162671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.165304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.186998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 960/1883] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:52.186998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.193652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1b6d8e84-d517-4ca3-bf7d-f07d85890033 None None] MFA Rules not processed for user `61af4255e31c4c7a8d523487f123e345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:52.195605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1b6d8e84-d517-4ca3-bf7d-f07d85890033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:52.202824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474ea7e-574b-4c11-9cae-322fc2b090fe admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.213504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.214827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.215338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.215780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.217035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.220021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.250548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 961/1884] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:52.250548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.274415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3f255-ea88-4f8a-96a7-c862f3915ba6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.286282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.288737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.289411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.290045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.291385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.294949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.327104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 962/1885] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:52.327104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.357816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 924/1886] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:52.357816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8e84-d517-4ca3-bf7d-f07d85890033 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.358117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06407f26-bef0-41ea-8c1c-763c93ecd3b2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.367213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.368858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.369737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.370675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] PATH_INFO: `/v3/projects/22a1cfc43c4b46efb324035b656bbf16/users/b27a97f4b14e4c258145cabb7730f1d0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.391175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8e84-d517-4ca3-bf7d-f07d85890033 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.398908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.399958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.400069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a1cfc43c4b46efb324035b656bbf16, user_id=b27a97f4b14e4c258145cabb7730f1d0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.400726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.401302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] PATH_INFO: `/v3/projects/228cd06a4f29456f8ee47405af64c5f7/users/03bbcccf4fe94846a3739108f9870ce7/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.405597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.427219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 963/1887] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:52 2022] PUT /identity/v3/projects/22a1cfc43c4b46efb324035b656bbf16/users/b27a97f4b14e4c258145cabb7730f1d0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:52.430408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.442293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=228cd06a4f29456f8ee47405af64c5f7, user_id=03bbcccf4fe94846a3739108f9870ce7, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.448879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.483225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 925/1888] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:52 2022] PUT /identity/v3/projects/228cd06a4f29456f8ee47405af64c5f7/users/03bbcccf4fe94846a3739108f9870ce7/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:52.484508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.519822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:52.535986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd29c4-a88e-45a5-b547-5de2bd392782 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.600819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb77f-29c7-4045-b6cd-ef953d5ecb1f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.608337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922c715-491a-4adf-9aae-4cdfb091e586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.609503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922c715-491a-4adf-9aae-4cdfb091e586 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.609998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922c715-491a-4adf-9aae-4cdfb091e586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.610488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922c715-491a-4adf-9aae-4cdfb091e586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.611629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922c715-491a-4adf-9aae-4cdfb091e586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.614450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922c715-491a-4adf-9aae-4cdfb091e586 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.630432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.631055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.631394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.631661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.632299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.635128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.641389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 926/1889] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:52.641389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654deb2-1a89-437a-abca-33047a8fa793 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.641671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654deb2-1a89-437a-abca-33047a8fa793 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.642117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654deb2-1a89-437a-abca-33047a8fa793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.703188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 964/1890] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:57:52 2022] POST /identity/v3/projects => generated 369 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:52.703188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.707271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d654deb2-1a89-437a-abca-33047a8fa793 None None] MFA Rules not processed for user `03bbcccf4fe94846a3739108f9870ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:52.708521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d654deb2-1a89-437a-abca-33047a8fa793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:52.744016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:52.798963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b8a5-69d4-45cc-89a9-21419cc61fa7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.803951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.805343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.805633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.805914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] PATH_INFO: `/v3/projects/22a1cfc43c4b46efb324035b656bbf16/users/b27a97f4b14e4c258145cabb7730f1d0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.809741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 927/1891] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:52.811002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a1cfc43c4b46efb324035b656bbf16, user_id=b27a97f4b14e4c258145cabb7730f1d0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.811273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.811541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.811805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.814143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.829886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 965/1892] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:52 2022] PUT /identity/v3/projects/22a1cfc43c4b46efb324035b656bbf16/users/b27a97f4b14e4c258145cabb7730f1d0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:52.830817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.851237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] MFA Rules not processed for user `ca360b149ac8402282733edfc4c67f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:52.852754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:52.926169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ef484-89e4-4a8a-9f30-faa3cb1d62da admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:52.932175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee86351-f69e-4b0c-9fc1-896eb6f3e0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:52.933814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86351-f69e-4b0c-9fc1-896eb6f3e0cb admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:52.934250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86351-f69e-4b0c-9fc1-896eb6f3e0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:52.934672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86351-f69e-4b0c-9fc1-896eb6f3e0cb admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:52.935501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee86351-f69e-4b0c-9fc1-896eb6f3e0cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:52.937886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee86351-f69e-4b0c-9fc1-896eb6f3e0cb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:52.949068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 928/1893] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:52.950047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:52.951381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:53.012634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 966/1894] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:57:52 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:57:53.014244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.014531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.014991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.024881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:53.059686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] MFA Rules not processed for user `b27a97f4b14e4c258145cabb7730f1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:53.061256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:53.068578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be35ec-abc4-4ac9-883e-f192fccb7fe3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.073270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.074452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.074722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.074993 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.140706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:53.167301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 967/1895] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:53.167301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.167301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:53.178980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.185638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.199351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 929/1896] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:53.200614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.212788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47508f3b-9125-46e1-b6fe-2cecca45a263 service glance] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.218459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.219390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.219883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.220260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.221006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.223047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.239590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 930/1897] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:53.240937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.246091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:53.276586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ffb2-f41c-44a9-8fe8-7d8aaf6b8abe None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.280664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.281585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.281585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.281850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.291303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.293311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.303838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 968/1898] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:53.304509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.316464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ccf38-1ff4-46f9-a8db-bb5353eb0473 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.321661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.322416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.322664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.322934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] PATH_INFO: `/v3/projects/5bab422a3730477d9741e83a5cb7b702/users/c111795324014cfea9b9366f70254fe0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.327126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2060d-42bd-4b02-938f-09877d24ae14 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.331363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.332061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.332413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.332755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.342746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bab422a3730477d9741e83a5cb7b702, user_id=c111795324014cfea9b9366f70254fe0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.344149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.345848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.346866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.369369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 931/1899] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:53.377375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 969/1900] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:53 2022] PUT /identity/v3/projects/5bab422a3730477d9741e83a5cb7b702/users/c111795324014cfea9b9366f70254fe0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:53.382427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.399251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497e0b-70e8-4f97-b4ed-b5e877e5f2e4 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.486784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24dd05-8399-488a-bbc0-ad9e3a1a776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.487482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24dd05-8399-488a-bbc0-ad9e3a1a776d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.487891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24dd05-8399-488a-bbc0-ad9e3a1a776d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.488109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24dd05-8399-488a-bbc0-ad9e3a1a776d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.488819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24dd05-8399-488a-bbc0-ad9e3a1a776d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.490596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24dd05-8399-488a-bbc0-ad9e3a1a776d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.507807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 932/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:57:53.511946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.551358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d117521-1b64-4f68-91c7-46620f396dbc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.581444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.582159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.582430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.582706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] PATH_INFO: `/v3/projects/5bab422a3730477d9741e83a5cb7b702/users/c111795324014cfea9b9366f70254fe0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.596952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bab422a3730477d9741e83a5cb7b702, user_id=c111795324014cfea9b9366f70254fe0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.599673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.631993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 970/1902] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:57:53 2022] PUT /identity/v3/projects/5bab422a3730477d9741e83a5cb7b702/users/c111795324014cfea9b9366f70254fe0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:53.633356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2099e-b467-4e21-84a2-96113e58049d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.633672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2099e-b467-4e21-84a2-96113e58049d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.633950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2099e-b467-4e21-84a2-96113e58049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.688578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4cc2099e-b467-4e21-84a2-96113e58049d None None] MFA Rules not processed for user `c111795324014cfea9b9366f70254fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:53.691295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4cc2099e-b467-4e21-84a2-96113e58049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:53.798759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.804282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:53.817910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 933/1903] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:53.869895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a784e-47ca-40ba-8122-39c3b5017bbf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:53.875595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:53.876429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:53.876898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:53.877337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:53.886506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:53.949746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:53.951198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:53.962716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 971/1904] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:57:53.988971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2099e-b467-4e21-84a2-96113e58049d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:53.994905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2099e-b467-4e21-84a2-96113e58049d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:54.056149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805b684-ed3e-4b4e-878d-1def9ec85116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:54.060591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805b684-ed3e-4b4e-878d-1def9ec85116 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:54.060769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805b684-ed3e-4b4e-878d-1def9ec85116 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:54.063105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805b684-ed3e-4b4e-878d-1def9ec85116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:54.085431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805b684-ed3e-4b4e-878d-1def9ec85116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:54.087428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805b684-ed3e-4b4e-878d-1def9ec85116 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:54.105752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 934/1905] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:54.288688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:54.299849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782c56c-902e-4269-a726-06da2be1a66c service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:54.310605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:54.311433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:54.311808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:54.312151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:54.449504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:54.451052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:54.463183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 972/1906] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:54.912872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5fdc-9962-414b-8ad2-be1df1bf6d18 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:54.913440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5fdc-9962-414b-8ad2-be1df1bf6d18 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:54.913745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5fdc-9962-414b-8ad2-be1df1bf6d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:54.959895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-68be5fdc-9962-414b-8ad2-be1df1bf6d18 None None] MFA Rules not processed for user `ca360b149ac8402282733edfc4c67f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:54.962009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-68be5fdc-9962-414b-8ad2-be1df1bf6d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:55.089409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 935/1907] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:58.766238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:58.843753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbf532-ddd0-4622-b249-4e09c942fefa service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:58.849703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:58.850437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:58.850931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:58.851306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] PATH_INFO: `/v3/users/61af4255e31c4c7a8d523487f123e345` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:58.852514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61af4255e31c4c7a8d523487f123e345)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:58.854575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:58.908191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61af4255e31c4c7a8d523487f123e345'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:58.910354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:58.918061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61af4255e31c4c7a8d523487f123e345'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:58.936064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61af4255e31c4c7a8d523487f123e345'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:58.936802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61af4255e31c4c7a8d523487f123e345'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:58.945760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61af4255e31c4c7a8d523487f123e345'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:58.952640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 973/1908] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:58 2022] DELETE /identity/v3/users/61af4255e31c4c7a8d523487f123e345 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:58.957159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:58.957690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:58.957781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:58.980779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:57:58.982073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:59.045543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 936/1909] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:57:59.060661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:59.062114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:59.130163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f252f-d143-46cd-850f-a879a0f0d165 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:59.134331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:59.134941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:59.135213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:59.135484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:59.148996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:59.155035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:59.167385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 974/1910] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:57:59.444527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:59.446490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:59.463278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f9d80-605f-4993-a426-04e423edb0f3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:59.470068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:59.471291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:59.471902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:59.472460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] PATH_INFO: `/v3/projects/e70b29cddf6f44689ae6b05afbf67dcd` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:59.475294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70b29cddf6f44689ae6b05afbf67dcd)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:59.481368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:59.598497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70b29cddf6f44689ae6b05afbf67dcd', 'request_id': 'req-9c455ded-4811-4d00-9d62-023a4a2e94b6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.607878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70b29cddf6f44689ae6b05afbf67dcd', 'request_id': 'req-9c455ded-4811-4d00-9d62-023a4a2e94b6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.617516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70b29cddf6f44689ae6b05afbf67dcd', 'request_id': 'req-9c455ded-4811-4d00-9d62-023a4a2e94b6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.644309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] The token cache is being invalidate because project e70b29cddf6f44689ae6b05afbf67dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:57:59.644672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.653612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.660150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 937/1911] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:57:59 2022] DELETE /identity/v3/projects/e70b29cddf6f44689ae6b05afbf67dcd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:59.660318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:59.687029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9150d6-194f-41e5-93b3-7a5d7f355f1b service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:59.819950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:57:59.821354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:57:59.826392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:59.826392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:59.826392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:59.826392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] PATH_INFO: `/v3/users/c111795324014cfea9b9366f70254fe0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:59.827874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c111795324014cfea9b9366f70254fe0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:59.829980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:59.846202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c455ded-4811-4d00-9d62-023a4a2e94b6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:57:59.853477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:57:59.854299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:59.855386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:59.855802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] PATH_INFO: `/v3/users/43baaf0e01ee4bc8b7ec5e94d6a743b5` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:57:59.857901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43baaf0e01ee4bc8b7ec5e94d6a743b5)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:57:59.860519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:57:59.867541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:59.892483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c111795324014cfea9b9366f70254fe0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.907615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c111795324014cfea9b9366f70254fe0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.923516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:57:59.925799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c111795324014cfea9b9366f70254fe0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.927780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c111795324014cfea9b9366f70254fe0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.933981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43baaf0e01ee4bc8b7ec5e94d6a743b5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.937747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-51b088ad-ec5e-4282-a440-5dc39194d9d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c111795324014cfea9b9366f70254fe0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.945816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 975/1912] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:59 2022] DELETE /identity/v3/users/c111795324014cfea9b9366f70254fe0 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:57:59.949119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43baaf0e01ee4bc8b7ec5e94d6a743b5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.964949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43baaf0e01ee4bc8b7ec5e94d6a743b5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.980994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43baaf0e01ee4bc8b7ec5e94d6a743b5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.982111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43baaf0e01ee4bc8b7ec5e94d6a743b5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:57:59.990772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 938/1913] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:57:59 2022] DELETE /identity/v3/users/43baaf0e01ee4bc8b7ec5e94d6a743b5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:57:59.995973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f68e39-1a70-4813-974e-a168792015db None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:57:59.996328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f68e39-1a70-4813-974e-a168792015db None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:57:59.996689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f68e39-1a70-4813-974e-a168792015db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.042452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a2f68e39-1a70-4813-974e-a168792015db None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:00.044223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a2f68e39-1a70-4813-974e-a168792015db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:00.130793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 976/1914] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:00.142464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:00.144153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:00.213720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec159de2-c4e8-44e8-b008-60c9130c3e38 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:00.219137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44655a-aceb-407c-af86-28eeca3207fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:00.219763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44655a-aceb-407c-af86-28eeca3207fa service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.220044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44655a-aceb-407c-af86-28eeca3207fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.220315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44655a-aceb-407c-af86-28eeca3207fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.230730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44655a-aceb-407c-af86-28eeca3207fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:00.232171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44655a-aceb-407c-af86-28eeca3207fa service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:00.249009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 939/1915] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:00.308906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f68e39-1a70-4813-974e-a168792015db None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:00.310340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a2f68e39-1a70-4813-974e-a168792015db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:00.322079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f68e39-1a70-4813-974e-a168792015db None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:00.327244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:00.327868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.328759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.329037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] PATH_INFO: `/v3/projects/5bab422a3730477d9741e83a5cb7b702` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.330222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bab422a3730477d9741e83a5cb7b702)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:00.332134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:00.428101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bab422a3730477d9741e83a5cb7b702', 'request_id': 'req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:00.438665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bab422a3730477d9741e83a5cb7b702', 'request_id': 'req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:00.452871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bab422a3730477d9741e83a5cb7b702', 'request_id': 'req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:00.482567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] The token cache is being invalidate because project 5bab422a3730477d9741e83a5cb7b702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:00.483005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:00.483901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ce9366c-cdc5-426f-8e39-4dcdbfdb8832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:00.485964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 977/1916] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:00 2022] DELETE /identity/v3/projects/5bab422a3730477d9741e83a5cb7b702 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:00.491150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929c3c1-2279-49b6-802f-f7778c97f7a6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.491435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929c3c1-2279-49b6-802f-f7778c97f7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.491763 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929c3c1-2279-49b6-802f-f7778c97f7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.516773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1929c3c1-2279-49b6-802f-f7778c97f7a6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:00.518014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1929c3c1-2279-49b6-802f-f7778c97f7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:00.627774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 940/1917] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:00.636444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d3661-0dc5-467e-b22a-54fbe3bfe13d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.636751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d3661-0dc5-467e-b22a-54fbe3bfe13d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.637002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d3661-0dc5-467e-b22a-54fbe3bfe13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.657988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-eb4d3661-0dc5-467e-b22a-54fbe3bfe13d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:00.659428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-eb4d3661-0dc5-467e-b22a-54fbe3bfe13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:00.667586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 978/1918] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:00.672181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929c3c1-2279-49b6-802f-f7778c97f7a6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:00.687872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929c3c1-2279-49b6-802f-f7778c97f7a6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:00.692476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:00.693081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.693387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.693656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.694494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:00.694658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:00.696271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:00.713256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 941/1919] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:00.715625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d3661-0dc5-467e-b22a-54fbe3bfe13d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:00.726997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d3661-0dc5-467e-b22a-54fbe3bfe13d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:00.732051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:00.732809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.733189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.733585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.734545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:00.736619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:00.771564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 979/1920] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:00 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:00.772707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:00.783376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e38a5-dbdf-40a0-aa60-03a495c59e20 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:00.798362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:00.877698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:00.880075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:00.896373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:00.897136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.898924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.899442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.900320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:00.903499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:00.937307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 942/1921] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:00 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:00.939695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:00.955286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e8583-0506-4ed3-af99-e38451121c71 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:00.965390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:00.966015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:00.966267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:00.966701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:00.967603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:00.969402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:00.977520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:00.985663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 943/1922] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:00.990307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.001943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee250da-788b-4cb0-a372-06aeebcff135 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.006454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba6075-a23a-4c1e-81e3-40c344fd24c1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.008530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.009908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.010255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.010551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.011744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.013527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.017828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.018487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.018779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.019045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] PATH_INFO: `/v3/projects/3eec7115e55640c3ba320071a7352a70` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.020206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eec7115e55640c3ba320071a7352a70)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.022180 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.028052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 944/1923] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:01.031696 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.044774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9cde0-42eb-4973-a95c-52586fec4479 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.051826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aca637-fc23-477a-8d8b-344afbe2f5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.053424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca637-fc23-477a-8d8b-344afbe2f5e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.053866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca637-fc23-477a-8d8b-344afbe2f5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.054235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca637-fc23-477a-8d8b-344afbe2f5e5 admin admin] PATH_INFO: `/v3/projects/f041a001be8c4d8a8389cf2581853eb7/users/4881ae33cec044dd96942f1b27357142/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.077065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aca637-fc23-477a-8d8b-344afbe2f5e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f041a001be8c4d8a8389cf2581853eb7, user_id=4881ae33cec044dd96942f1b27357142, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.081406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aca637-fc23-477a-8d8b-344afbe2f5e5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.095937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eec7115e55640c3ba320071a7352a70', 'request_id': 'req-44e3749e-4108-4db1-9a29-bb2d6dba4453'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.103350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 945/1924] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:01 2022] PUT /identity/v3/projects/f041a001be8c4d8a8389cf2581853eb7/users/4881ae33cec044dd96942f1b27357142/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:01.110364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eec7115e55640c3ba320071a7352a70', 'request_id': 'req-44e3749e-4108-4db1-9a29-bb2d6dba4453'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.116941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.117470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.117910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.118994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eec7115e55640c3ba320071a7352a70', 'request_id': 'req-44e3749e-4108-4db1-9a29-bb2d6dba4453'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.141615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] The token cache is being invalidate because project 3eec7115e55640c3ba320071a7352a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:01.142087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.142963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.146148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 980/1925] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:00 2022] DELETE /identity/v3/projects/3eec7115e55640c3ba320071a7352a70 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:01.152306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.153786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:01.161275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] MFA Rules not processed for user `4881ae33cec044dd96942f1b27357142`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:01.162733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:01.230072 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3749e-4108-4db1-9a29-bb2d6dba4453 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.252218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 946/1926] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:01.257828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.263306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:01.309334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.310135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.310467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.310792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] PATH_INFO: `/v3/users/b27a97f4b14e4c258145cabb7730f1d0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.312587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b27a97f4b14e4c258145cabb7730f1d0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.314630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.327191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:01.350734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c9369-0e9c-4b01-b40e-f3ec1f1a2394 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.352335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:01.356859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.357516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.357795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.358063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.359178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.361742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.410627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 947/1927] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:01 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:01.414626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.424347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b27a97f4b14e4c258145cabb7730f1d0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.452721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b27a97f4b14e4c258145cabb7730f1d0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.470557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b27a97f4b14e4c258145cabb7730f1d0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.471432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b27a97f4b14e4c258145cabb7730f1d0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.484010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b27a97f4b14e4c258145cabb7730f1d0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:01.491869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 981/1928] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:01 2022] DELETE /identity/v3/users/b27a97f4b14e4c258145cabb7730f1d0 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:01.508558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2c6c-179c-4b93-b16a-634d2a4d1d34 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.586084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.586779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.587145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.587402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.588125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.590103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.625536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 948/1929] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:01 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:01.626405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.630223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:01.651995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9e84f-2689-4331-9180-ee51bfcbe5a4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.685973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.690483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.692047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.692334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.695732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.697437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.719159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 982/1930] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:01.723075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.742913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38252e62-9bd8-45ba-979b-f77935fe8274 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.752123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.752792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.753067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.753364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] PATH_INFO: `/v3/projects/5bbbb993704a4a7abf8760d66aea41e9/users/aacd4e0c50354c91a6cfc3e7cef6899b/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.773377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbbb993704a4a7abf8760d66aea41e9, user_id=aacd4e0c50354c91a6cfc3e7cef6899b, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.777175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.797496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 949/1931] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:01 2022] PUT /identity/v3/projects/5bbbb993704a4a7abf8760d66aea41e9/users/aacd4e0c50354c91a6cfc3e7cef6899b/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:01.799023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.811665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc88aa5-5f1d-4f0f-bf67-7a4131b6fe54 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.858903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.860838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:01.899258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.900092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.902429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.902429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.903501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.906196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.919796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 983/1932] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:01.924972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:01.941697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:01.942638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54565ad-a5ff-4f53-a0af-3005067a3cb7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.947847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d34bb-52d5-475f-8437-eb242bb3691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:01.948605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d34bb-52d5-475f-8437-eb242bb3691a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.949055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d34bb-52d5-475f-8437-eb242bb3691a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.949317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d34bb-52d5-475f-8437-eb242bb3691a admin admin] PATH_INFO: `/v3/projects/5bbbb993704a4a7abf8760d66aea41e9/users/aacd4e0c50354c91a6cfc3e7cef6899b/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:01.951482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d34bb-52d5-475f-8437-eb242bb3691a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbbb993704a4a7abf8760d66aea41e9, user_id=aacd4e0c50354c91a6cfc3e7cef6899b, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:01.954268 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d34bb-52d5-475f-8437-eb242bb3691a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:01.966049 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd3473-5031-42a7-9426-f16671b34e3f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:01.966727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 984/1933] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:01 2022] PUT /identity/v3/projects/5bbbb993704a4a7abf8760d66aea41e9/users/aacd4e0c50354c91a6cfc3e7cef6899b/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:01.976861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:01.977062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:01.977378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.038021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] MFA Rules not processed for user `aacd4e0c50354c91a6cfc3e7cef6899b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:02.040189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.050475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.051130 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.051415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.051694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] PATH_INFO: `/v3/projects/22a1cfc43c4b46efb324035b656bbf16` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.052986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a1cfc43c4b46efb324035b656bbf16)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.054985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.116205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a1cfc43c4b46efb324035b656bbf16', 'request_id': 'req-7d07bd76-0890-4341-8133-3f652941d026'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.124285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a1cfc43c4b46efb324035b656bbf16', 'request_id': 'req-7d07bd76-0890-4341-8133-3f652941d026'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.127421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:02.129890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a1cfc43c4b46efb324035b656bbf16', 'request_id': 'req-7d07bd76-0890-4341-8133-3f652941d026'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.139661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 985/1934] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:02.152809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] The token cache is being invalidate because project 22a1cfc43c4b46efb324035b656bbf16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:02.153275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.153957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.154132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d07bd76-0890-4341-8133-3f652941d026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.155337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.159660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 950/1935] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:01 2022] DELETE /identity/v3/projects/22a1cfc43c4b46efb324035b656bbf16 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:02.161779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39898858-be9c-4e2e-bb1b-2ec35d625ba3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.162197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39898858-be9c-4e2e-bb1b-2ec35d625ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.162770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39898858-be9c-4e2e-bb1b-2ec35d625ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.187516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-39898858-be9c-4e2e-bb1b-2ec35d625ba3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:02.189488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-39898858-be9c-4e2e-bb1b-2ec35d625ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.221818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c2f8e-378c-40a3-836c-022c2314379d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:02.293251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 951/1936] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:02.300332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190c81f-f8e0-42f3-b456-6cedbbc8cb01 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.300924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190c81f-f8e0-42f3-b456-6cedbbc8cb01 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.301402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190c81f-f8e0-42f3-b456-6cedbbc8cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.313139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.313828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.314125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.314392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.316571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.322697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3190c81f-f8e0-42f3-b456-6cedbbc8cb01 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:02.324614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3190c81f-f8e0-42f3-b456-6cedbbc8cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.335221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 952/1937] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:02.341108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190c81f-f8e0-42f3-b456-6cedbbc8cb01 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.351773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190c81f-f8e0-42f3-b456-6cedbbc8cb01 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:02.358296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.359034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.359409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.359794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.360540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:02.360935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.362919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.395779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 953/1938] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:02.397777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.407808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:02.410002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.414141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.418601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679caf6-a038-47e8-b9aa-25573325dd6e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:02.429147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.431115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.432028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.433230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.434134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.437125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 986/1939] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:02.438571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.541727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 954/1940] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:02 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:02.547332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.559379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.568004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.571485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ef3e0-97b8-440c-ba0b-1fba6564cecb service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:02.659730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.660361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.660633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.660897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.661577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.663788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.675037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed965df-2fe8-4271-8307-70471e79c521 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:02.680096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.680735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.681019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.681339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] PATH_INFO: `/v3/users/4881ae33cec044dd96942f1b27357142` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.683182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4881ae33cec044dd96942f1b27357142)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.683599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:02.685106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.710993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 987/1941] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:02 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:02.711658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.745758 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca8e8d-1d43-4b50-bc72-4d9c394aa02e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:02.754801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4881ae33cec044dd96942f1b27357142'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.758211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.759110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.759534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.760052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.763226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.774156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4881ae33cec044dd96942f1b27357142'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.775080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.785830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4881ae33cec044dd96942f1b27357142'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.800055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 988/1942] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:02.806334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.814659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4881ae33cec044dd96942f1b27357142'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.817440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ea20e837-3d67-4f8e-8621-b5fc58d065d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4881ae33cec044dd96942f1b27357142'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:02.845111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 955/1943] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:02 2022] DELETE /identity/v3/users/4881ae33cec044dd96942f1b27357142 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:02.853142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.854012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.854645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.860147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9802268-d0c8-470c-9137-80ad5b521b04 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:02.906665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:02.909401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.965559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:02.966901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:02.967323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:02.968056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:02.969231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:02.971364 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:02.990281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 989/1944] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:02.991985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:02.994536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:02.999376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:03.015485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc2e3b-c24b-41b5-9f69-dbb4d6c65fdb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.020439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 956/1945] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:03.025245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21116a5c-a9c1-4ae3-8f98-6b26f7013e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.026581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116a5c-a9c1-4ae3-8f98-6b26f7013e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.027014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116a5c-a9c1-4ae3-8f98-6b26f7013e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.028150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116a5c-a9c1-4ae3-8f98-6b26f7013e18 admin admin] PATH_INFO: `/v3/projects/b8d03d510c7846f3bedc7b6a86aea819/users/305294dd09944e5088157ede91a30e25/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.059978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21116a5c-a9c1-4ae3-8f98-6b26f7013e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d03d510c7846f3bedc7b6a86aea819, user_id=305294dd09944e5088157ede91a30e25, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.064992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.066970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21116a5c-a9c1-4ae3-8f98-6b26f7013e18 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.072611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:03.101697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 990/1946] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:02 2022] PUT /identity/v3/projects/b8d03d510c7846f3bedc7b6a86aea819/users/305294dd09944e5088157ede91a30e25/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:03.113626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1f89-2fe3-44bd-847f-7c710b2ab7fa None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.113914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1f89-2fe3-44bd-847f-7c710b2ab7fa None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.114277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f1f89-2fe3-44bd-847f-7c710b2ab7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.143902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-201f1f89-2fe3-44bd-847f-7c710b2ab7fa None None] MFA Rules not processed for user `305294dd09944e5088157ede91a30e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:03.145316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-201f1f89-2fe3-44bd-847f-7c710b2ab7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:03.173858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b44d79-437b-4ff8-97b8-5ba31bad04a8 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.229121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 991/1947] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:03.234241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f1f89-2fe3-44bd-847f-7c710b2ab7fa None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.271715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.272195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.272583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.272966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.298980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:03.326721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f1f89-2fe3-44bd-847f-7c710b2ab7fa None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.333384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.334276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.335120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.335622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.337065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.339747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.368208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.369830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.384337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 992/1948] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:03 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:03.385031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.392056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 957/1949] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:03.470363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361124-5d64-4a10-8b09-5870a91ed925 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.477908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.479023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.480417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.480964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.481804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.485971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.517955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 993/1950] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:03 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:03.520466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.532755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a345-1ea1-4633-afa7-88a302b0022f service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.539450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.540232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.540925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.541280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.542953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.544878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.564575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 958/1951] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:03.567646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.583145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6232522-6238-4f2c-b99e-30227dd964e9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.589778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.590451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.590771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.591050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] PATH_INFO: `/v3/projects/1a5981630fbb4246a4df7dfe16b0cdd2/users/b01d803e0e774637b60a02eeb2cf57e2/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.608881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5981630fbb4246a4df7dfe16b0cdd2, user_id=b01d803e0e774637b60a02eeb2cf57e2, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.612612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.635932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 994/1952] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:03 2022] PUT /identity/v3/projects/1a5981630fbb4246a4df7dfe16b0cdd2/users/b01d803e0e774637b60a02eeb2cf57e2/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:03.637692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.651098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2253b2-eebd-4f52-8b4b-16d3c33d4481 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.714831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:03.727258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2935070-f19d-4570-aec1-d130f215a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.728027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2935070-f19d-4570-aec1-d130f215a7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.728374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2935070-f19d-4570-aec1-d130f215a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.728718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2935070-f19d-4570-aec1-d130f215a7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.729496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2935070-f19d-4570-aec1-d130f215a7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.731730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2935070-f19d-4570-aec1-d130f215a7d9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.744843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 959/1953] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:03.748402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.764609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66518b46-27a5-4213-8f71-1f4391013d15 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:03.773867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:03.774560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.774917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.775263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] PATH_INFO: `/v3/projects/1a5981630fbb4246a4df7dfe16b0cdd2/users/b01d803e0e774637b60a02eeb2cf57e2/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.778585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5981630fbb4246a4df7dfe16b0cdd2, user_id=b01d803e0e774637b60a02eeb2cf57e2, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:03.781563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:03.793875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 995/1954] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:03 2022] PUT /identity/v3/projects/1a5981630fbb4246a4df7dfe16b0cdd2/users/b01d803e0e774637b60a02eeb2cf57e2/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:03.804325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:03.804629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:03.804887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:03.838081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] MFA Rules not processed for user `b01d803e0e774637b60a02eeb2cf57e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:03.840026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:03.974002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 960/1955] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:03.985365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:03.988258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:04.066703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957956e-1c17-45ff-bafe-fc1558a393be admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:04.070658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:04.071292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:04.071592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:04.073728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:04.112624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:04.144631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:04.146126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:04.157337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 996/1956] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:04.892238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:04.893795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:04.987064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbe4c-3e27-4b88-86f6-e9be8b630094 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:04.992632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:04.993296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:04.993632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:04.993890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] PATH_INFO: `/v3/projects/f041a001be8c4d8a8389cf2581853eb7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:04.995606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f041a001be8c4d8a8389cf2581853eb7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:04.997603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:05.071322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f041a001be8c4d8a8389cf2581853eb7', 'request_id': 'req-7858063f-9dc9-4f76-92c2-21e46da1cc8d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.079193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f041a001be8c4d8a8389cf2581853eb7', 'request_id': 'req-7858063f-9dc9-4f76-92c2-21e46da1cc8d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.086718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f041a001be8c4d8a8389cf2581853eb7', 'request_id': 'req-7858063f-9dc9-4f76-92c2-21e46da1cc8d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.104232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] The token cache is being invalidate because project f041a001be8c4d8a8389cf2581853eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:05.105548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.106356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.108296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 961/1957] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:04 2022] DELETE /identity/v3/projects/f041a001be8c4d8a8389cf2581853eb7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:05.112207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:05.126165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f56d3-cccf-4015-8d5e-20e3100dedb7 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:05.201443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:05.202298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:05.202698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:05.203062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] PATH_INFO: `/v3/users/aacd4e0c50354c91a6cfc3e7cef6899b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:05.204379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacd4e0c50354c91a6cfc3e7cef6899b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:05.206574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:05.277824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacd4e0c50354c91a6cfc3e7cef6899b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.285038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacd4e0c50354c91a6cfc3e7cef6899b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.297703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacd4e0c50354c91a6cfc3e7cef6899b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.298450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacd4e0c50354c91a6cfc3e7cef6899b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.308745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f07985eb-fe82-4b81-8237-38c31f1cc8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacd4e0c50354c91a6cfc3e7cef6899b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.315792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 997/1958] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:05 2022] DELETE /identity/v3/users/aacd4e0c50354c91a6cfc3e7cef6899b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:05.672645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:05.674468 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:05.690483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858063f-9dc9-4f76-92c2-21e46da1cc8d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:05.694884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:05.830565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:05.831229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:05.831508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:05.831781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] PATH_INFO: `/v3/projects/5bbbb993704a4a7abf8760d66aea41e9` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:05.841410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bbbb993704a4a7abf8760d66aea41e9)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:05.843541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:05.926253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bbbb993704a4a7abf8760d66aea41e9', 'request_id': 'req-a1f00701-8de6-40dc-a58b-11df124aad33'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.936286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bbbb993704a4a7abf8760d66aea41e9', 'request_id': 'req-a1f00701-8de6-40dc-a58b-11df124aad33'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.947072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bbbb993704a4a7abf8760d66aea41e9', 'request_id': 'req-a1f00701-8de6-40dc-a58b-11df124aad33'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.970999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] The token cache is being invalidate because project 5bbbb993704a4a7abf8760d66aea41e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:05.971412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.972086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a1f00701-8de6-40dc-a58b-11df124aad33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:05.974795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 962/1959] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:05 2022] DELETE /identity/v3/projects/5bbbb993704a4a7abf8760d66aea41e9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:05.979390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c92e0-8c13-4349-83ed-2b4e388e8d65 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:05.980023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c92e0-8c13-4349-83ed-2b4e388e8d65 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:05.980470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c92e0-8c13-4349-83ed-2b4e388e8d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.036079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4c3c92e0-8c13-4349-83ed-2b4e388e8d65 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:06.037438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4c3c92e0-8c13-4349-83ed-2b4e388e8d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:06.162056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:06.180636 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 998/1960] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:06.190935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f94a4-4771-442f-8581-0e03e1c4253a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.191512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f94a4-4771-442f-8581-0e03e1c4253a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.192089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f94a4-4771-442f-8581-0e03e1c4253a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.250232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-468f94a4-4771-442f-8581-0e03e1c4253a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:06.251577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-468f94a4-4771-442f-8581-0e03e1c4253a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:06.263766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 963/1961] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:06.264893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c92e0-8c13-4349-83ed-2b4e388e8d65 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.280604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c92e0-8c13-4349-83ed-2b4e388e8d65 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.298715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.300202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.301623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.302663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.305282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:06.305856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.311293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:06.343638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f94a4-4771-442f-8581-0e03e1c4253a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.353355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 999/1962] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:58:06.376024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f94a4-4771-442f-8581-0e03e1c4253a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.382087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.382714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.382991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.383258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.383890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.386040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:06.418154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 964/1963] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:06 2022] POST /identity/v3/projects => generated 391 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:06.423815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.437498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6567a-f895-4125-811d-d9b9e66b9a16 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.501099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.501752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.502049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.502311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.502938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.505010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:06.538538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1000/1964] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:06 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:06.542935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.559613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca7a87-6dd1-4943-b7f3-9a4fe0c87d00 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.564655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.565312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.565592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.565863 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.566514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.568175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:06.581168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 965/1965] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:06.585036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.595891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331728-1345-401c-83e5-021dddd634a7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.600837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423ee4f-b18b-413e-984f-900423138b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.601434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423ee4f-b18b-413e-984f-900423138b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.601715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423ee4f-b18b-413e-984f-900423138b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.601989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423ee4f-b18b-413e-984f-900423138b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.602667 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423ee4f-b18b-413e-984f-900423138b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.604288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423ee4f-b18b-413e-984f-900423138b9f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:06.616050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1001/1966] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:06.620230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.630387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bbf06-0508-4d62-a3f8-f55d948f0eff admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.635599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.636524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.636802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.637075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] PATH_INFO: `/v3/projects/9f143e50be85490bbc174e761a92ea21/users/d0a094b7997142888ebf04093f003222/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.653456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f143e50be85490bbc174e761a92ea21, user_id=d0a094b7997142888ebf04093f003222, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.656104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:06.677395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 966/1967] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:06 2022] PUT /identity/v3/projects/9f143e50be85490bbc174e761a92ea21/users/d0a094b7997142888ebf04093f003222/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:06.681469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0b7b0-7012-4c0f-95db-a829c481bd9d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.681757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0b7b0-7012-4c0f-95db-a829c481bd9d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.682030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0b7b0-7012-4c0f-95db-a829c481bd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.711719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-aba0b7b0-7012-4c0f-95db-a829c481bd9d None None] MFA Rules not processed for user `d0a094b7997142888ebf04093f003222`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:06.712971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aba0b7b0-7012-4c0f-95db-a829c481bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:06.779736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1002/1968] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:06.783858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.818777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:06.850333 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfd4c-a7ae-4f43-819c-f2be2c642516 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.855133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.855680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.855957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.856226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.856856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.858904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:06.889368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 967/1969] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:06 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:06.892080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0b7b0-7012-4c0f-95db-a829c481bd9d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:06.908522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0b7b0-7012-4c0f-95db-a829c481bd9d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:06.963438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:06.976019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:06.976693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:06.976974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:06.977264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:06.977910 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:06.980087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:07.009160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1003/1970] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:06 2022] POST /identity/v3/users => generated 302 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:07.014009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:07.024927 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2bbf6-c62f-4980-b340-c787c5e62b8a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:07.031232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:07.031945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:07.032288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:07.032625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:07.033418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:07.035232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:07.046932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 968/1971] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:07.051335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:07.062756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d8690-c381-4c11-be37-849028123485 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:07.070417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:07.071093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:07.071371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:07.071644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] PATH_INFO: `/v3/projects/38cd92a2092e45af85dcd067ec87209d/users/4f906e15ec144a7db784c742618ec7bb/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:07.089774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cd92a2092e45af85dcd067ec87209d, user_id=4f906e15ec144a7db784c742618ec7bb, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:07.092561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:07.109124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1004/1972] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:07 2022] PUT /identity/v3/projects/38cd92a2092e45af85dcd067ec87209d/users/4f906e15ec144a7db784c742618ec7bb/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:07.113136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:07.123697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf708ee-8479-4395-bab0-583c70b01e51 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:07.190822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dad9eb-74dd-4dea-8300-e904c560746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:07.191538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dad9eb-74dd-4dea-8300-e904c560746c admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:07.191885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dad9eb-74dd-4dea-8300-e904c560746c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:07.192164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dad9eb-74dd-4dea-8300-e904c560746c admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:07.192815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dad9eb-74dd-4dea-8300-e904c560746c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:07.194497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dad9eb-74dd-4dea-8300-e904c560746c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:07.211327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 969/1973] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:07.215334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:07.226700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0202166-409d-41e4-95ed-1b0cb139ff72 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:07.231558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:07.232205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:07.232486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:07.232753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] PATH_INFO: `/v3/projects/38cd92a2092e45af85dcd067ec87209d/users/4f906e15ec144a7db784c742618ec7bb/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:07.234691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cd92a2092e45af85dcd067ec87209d, user_id=4f906e15ec144a7db784c742618ec7bb, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:07.250169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:07.261789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1005/1974] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:07 2022] PUT /identity/v3/projects/38cd92a2092e45af85dcd067ec87209d/users/4f906e15ec144a7db784c742618ec7bb/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:07.272525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532bcf-54ba-4513-b9e3-3c7b3db40f77 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:07.272955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532bcf-54ba-4513-b9e3-3c7b3db40f77 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:07.273411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532bcf-54ba-4513-b9e3-3c7b3db40f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:07.304566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f8532bcf-54ba-4513-b9e3-3c7b3db40f77 None None] MFA Rules not processed for user `4f906e15ec144a7db784c742618ec7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:07.306220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f8532bcf-54ba-4513-b9e3-3c7b3db40f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:07.407788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 970/1975] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:07.429352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:07.431346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:07.514631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cc5dc-b5d4-4e32-a9c8-219badab4cac admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:07.518448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:07.519069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:07.519356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:07.519626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:07.531007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:07.532495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:07.543026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1006/1976] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:07.592420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8532bcf-54ba-4513-b9e3-3c7b3db40f77 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:07.601935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8532bcf-54ba-4513-b9e3-3c7b3db40f77 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:07.606203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:07.606836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:07.607117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:07.607522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:07.650903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:07.685183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:07.686691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:07.695910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 971/1977] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:08.920120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:08.921679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:08.933461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c365dd-9c28-4197-b0f6-e4fbf57a37b1 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:08.937906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf01d21-3c72-4b36-bd14-393c03680b16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:08.938577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01d21-3c72-4b36-bd14-393c03680b16 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:08.938923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01d21-3c72-4b36-bd14-393c03680b16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:08.939263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01d21-3c72-4b36-bd14-393c03680b16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:08.941038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dbf01d21-3c72-4b36-bd14-393c03680b16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:09.006661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf01d21-3c72-4b36-bd14-393c03680b16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:09.008208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf01d21-3c72-4b36-bd14-393c03680b16 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:09.021541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1007/1978] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:11.226016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:11.297011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70d885-625b-416a-bdeb-08be0196d7c1 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:11.301640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:11.302294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:11.302571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:11.302853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] PATH_INFO: `/v3/users/d0a094b7997142888ebf04093f003222` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:11.303949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a094b7997142888ebf04093f003222)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:11.306097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:11.359428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a094b7997142888ebf04093f003222'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:11.373472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a094b7997142888ebf04093f003222'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:11.389284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a094b7997142888ebf04093f003222'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:11.398750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a094b7997142888ebf04093f003222'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:11.399633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a094b7997142888ebf04093f003222'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:11.410491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 972/1979] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:11 2022] DELETE /identity/v3/users/d0a094b7997142888ebf04093f003222 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:11.412294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:11.412738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:11.413017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:11.434131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:11.435965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:11.475330 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:11.507055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1008/1980] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:11.517123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:11.518569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:11.571934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:11.579030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c4a64-5dcb-4908-9257-978c46a946cf admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:11.583181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:11.583825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:11.584104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:11.584377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:11.594190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:11.595633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:11.608132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 973/1981] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:11.907022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:11.908893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:11.923595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b825f-5b79-4c14-95ca-b5a3a6ada2d5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:11.929681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:11.930435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:11.930783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:11.931094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] PATH_INFO: `/v3/projects/9f143e50be85490bbc174e761a92ea21` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:11.932207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f143e50be85490bbc174e761a92ea21)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:11.934733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:11.989595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f143e50be85490bbc174e761a92ea21', 'request_id': 'req-1a83f967-5918-4f21-acb5-b957e1284df6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:11.996473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f143e50be85490bbc174e761a92ea21', 'request_id': 'req-1a83f967-5918-4f21-acb5-b957e1284df6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.002197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f143e50be85490bbc174e761a92ea21', 'request_id': 'req-1a83f967-5918-4f21-acb5-b957e1284df6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.019465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] The token cache is being invalidate because project 9f143e50be85490bbc174e761a92ea21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:12.019762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.020413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.025245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1009/1982] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:11 2022] DELETE /identity/v3/projects/9f143e50be85490bbc174e761a92ea21 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:12.026779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:12.037101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f7da2-6e20-4577-924d-8cbd193f3c81 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:12.106524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:12.107148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:12.107584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:12.107890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] PATH_INFO: `/v3/users/4f906e15ec144a7db784c742618ec7bb` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:12.109125 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f906e15ec144a7db784c742618ec7bb)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:12.111086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:12.160914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f906e15ec144a7db784c742618ec7bb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.177498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f906e15ec144a7db784c742618ec7bb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.189120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f906e15ec144a7db784c742618ec7bb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.198891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f906e15ec144a7db784c742618ec7bb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.200819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-507823b4-1b1e-44ea-9f2a-597402faf53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f906e15ec144a7db784c742618ec7bb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.213362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 974/1983] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:12 2022] DELETE /identity/v3/users/4f906e15ec144a7db784c742618ec7bb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:12.537978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:12.539777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:12.564563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83f967-5918-4f21-acb5-b957e1284df6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:12.657039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:12.657714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:12.658019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:12.658347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] PATH_INFO: `/v3/projects/38cd92a2092e45af85dcd067ec87209d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:12.659479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38cd92a2092e45af85dcd067ec87209d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:12.661442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:12.716830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38cd92a2092e45af85dcd067ec87209d', 'request_id': 'req-f3570cbd-9e33-4ce9-8271-e9298118fcf7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.723413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38cd92a2092e45af85dcd067ec87209d', 'request_id': 'req-f3570cbd-9e33-4ce9-8271-e9298118fcf7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.730539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38cd92a2092e45af85dcd067ec87209d', 'request_id': 'req-f3570cbd-9e33-4ce9-8271-e9298118fcf7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.733168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:12.750848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] The token cache is being invalidate because project 38cd92a2092e45af85dcd067ec87209d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:12.751424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.752294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f3570cbd-9e33-4ce9-8271-e9298118fcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:12.753911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1010/1984] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:12 2022] DELETE /identity/v3/projects/38cd92a2092e45af85dcd067ec87209d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:12.759909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcd571-9667-4173-9a3c-872a771493de None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:12.760385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcd571-9667-4173-9a3c-872a771493de None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:12.760385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcd571-9667-4173-9a3c-872a771493de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:12.794659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-50bcd571-9667-4173-9a3c-872a771493de None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:12.796158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-50bcd571-9667-4173-9a3c-872a771493de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:12.917104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 975/1985] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:12.925502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8572f-632b-4bb5-98a5-a81d056f1841 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:12.925795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8572f-632b-4bb5-98a5-a81d056f1841 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:12.926074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8572f-632b-4bb5-98a5-a81d056f1841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:12.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d3c8572f-632b-4bb5-98a5-a81d056f1841 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:12.952067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d3c8572f-632b-4bb5-98a5-a81d056f1841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:12.965240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1011/1986] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:12.967083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcd571-9667-4173-9a3c-872a771493de None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:12.978868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcd571-9667-4173-9a3c-872a771493de None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:12.988969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:12.990161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:12.990647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:12.991104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.006152 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:13.006838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.027916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.040492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:13.057854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 976/1987] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:13.060000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8572f-632b-4bb5-98a5-a81d056f1841 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.079405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8572f-632b-4bb5-98a5-a81d056f1841 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.093463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.094304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.094588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.094892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.096339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.102235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.153875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.156383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:13.184079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719c29-fdd0-4549-ba79-a0623e51585c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.194631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.195491 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.195957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.197016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] PATH_INFO: `/v3/users/03bbcccf4fe94846a3739108f9870ce7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.200279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03bbcccf4fe94846a3739108f9870ce7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.203688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.209631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1012/1988] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:13 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:13.210634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.289787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03bbcccf4fe94846a3739108f9870ce7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:13.313472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03bbcccf4fe94846a3739108f9870ce7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:13.321579 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03bbcccf4fe94846a3739108f9870ce7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:13.327294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2674068-bd63-4bc7-9eeb-41996dad5426 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.331091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03bbcccf4fe94846a3739108f9870ce7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:13.332027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15f6f1ba-6110-4f17-84f0-a2275ea87c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03bbcccf4fe94846a3739108f9870ce7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:13.332624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.333455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.333798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.334252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.334951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.336963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.343728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 977/1989] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:13 2022] DELETE /identity/v3/users/03bbcccf4fe94846a3739108f9870ce7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:13.345056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.345556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.345837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.361101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1013/1990] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:13 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:13.365511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.367245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:13.369335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:13.371131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:13.382333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274781e1-8d3b-4427-8c01-9d44e6904d69 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.447555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 978/1991] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:13.458993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.459630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.459912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.460235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.460914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.462718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.462970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.465643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:13.477722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1014/1992] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:13.479418 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.491659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0084254-62f0-4df8-b802-0114260e3eca admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.499591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.500275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.501259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.501259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.501559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.503777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.521262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1015/1993] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:13.523879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.535095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a0b16-6c65-4bb1-8393-10c3eafb4b8b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.541179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.542446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.542722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.542998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] PATH_INFO: `/v3/projects/d2c488bf6f064a2f9e4796b0cbef0f19/users/517aac38e3b64b4dbef6f9278d82f3de/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.553097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30edcfb9-ae55-4498-8174-026f6a9bfa1e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.557306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357dfc1-c54b-41e2-8603-dd4e398ce9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.558502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dfc1-c54b-41e2-8603-dd4e398ce9ae service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.558821 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dfc1-c54b-41e2-8603-dd4e398ce9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.559040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dfc1-c54b-41e2-8603-dd4e398ce9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.568497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c488bf6f064a2f9e4796b0cbef0f19, user_id=517aac38e3b64b4dbef6f9278d82f3de, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.569760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357dfc1-c54b-41e2-8603-dd4e398ce9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.571932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.572468 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357dfc1-c54b-41e2-8603-dd4e398ce9ae service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.586698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 979/1994] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:13.588920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1016/1995] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:13 2022] PUT /identity/v3/projects/d2c488bf6f064a2f9e4796b0cbef0f19/users/517aac38e3b64b4dbef6f9278d82f3de/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:13.607307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fb5c4-f56a-4b0a-bc7a-a1e5c757eeef None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.607597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fb5c4-f56a-4b0a-bc7a-a1e5c757eeef None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.607873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fb5c4-f56a-4b0a-bc7a-a1e5c757eeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.634261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:13.644838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cf1fb5c4-f56a-4b0a-bc7a-a1e5c757eeef None None] MFA Rules not processed for user `517aac38e3b64b4dbef6f9278d82f3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:13.646350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cf1fb5c4-f56a-4b0a-bc7a-a1e5c757eeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:13.753661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 980/1996] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:13.763771 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.765584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:13.799958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:13.855187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7c0ff-556f-4766-a796-18b44316a5d6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:13.859822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:13.860722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:13.861130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:13.861537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:13.871151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:13.872718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:13.884046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1017/1997] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:13.914129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fb5c4-f56a-4b0a-bc7a-a1e5c757eeef None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.991168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:13.992567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.008509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fb5c4-f56a-4b0a-bc7a-a1e5c757eeef None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.014239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.014937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.015166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.015452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] PATH_INFO: `/v3/users/517aac38e3b64b4dbef6f9278d82f3de` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.016769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] RBAC: Authorizing `identity:delete_user(user_id=517aac38e3b64b4dbef6f9278d82f3de)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.018074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7b3df-144e-4d49-bfd7-a7c15f1273bb service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.018975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.027129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.027844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.028034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.028306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] PATH_INFO: `/v3/projects/228cd06a4f29456f8ee47405af64c5f7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.030465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228cd06a4f29456f8ee47405af64c5f7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.034298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.081867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517aac38e3b64b4dbef6f9278d82f3de'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.095228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517aac38e3b64b4dbef6f9278d82f3de'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.097171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228cd06a4f29456f8ee47405af64c5f7', 'request_id': 'req-5afb4071-5e3e-4553-95ca-d078c9af17a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.102788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517aac38e3b64b4dbef6f9278d82f3de'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.104056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228cd06a4f29456f8ee47405af64c5f7', 'request_id': 'req-5afb4071-5e3e-4553-95ca-d078c9af17a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.111050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228cd06a4f29456f8ee47405af64c5f7', 'request_id': 'req-5afb4071-5e3e-4553-95ca-d078c9af17a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.114578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517aac38e3b64b4dbef6f9278d82f3de'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.115195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ceee944-19db-4ddb-94e2-df810d23e51b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517aac38e3b64b4dbef6f9278d82f3de'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.122404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 981/1998] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:13 2022] DELETE /identity/v3/users/517aac38e3b64b4dbef6f9278d82f3de => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:14.129559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] The token cache is being invalidate because project 228cd06a4f29456f8ee47405af64c5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:14.129850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.130505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5afb4071-5e3e-4553-95ca-d078c9af17a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.132371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1018/1999] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:13 2022] DELETE /identity/v3/projects/228cd06a4f29456f8ee47405af64c5f7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:14.137235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34d475-b8e5-486e-95f5-22a5824c4270 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.137389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.137559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34d475-b8e5-486e-95f5-22a5824c4270 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.137655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.137810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34d475-b8e5-486e-95f5-22a5824c4270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.137919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.159039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5e34d475-b8e5-486e-95f5-22a5824c4270 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:14.160314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5e34d475-b8e5-486e-95f5-22a5824c4270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.160927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:14.162134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.174784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:14.225192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1019/2000] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:14.226095 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 982/2001] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:14.233176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.233808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.234127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.234415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.235061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.250606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978d897-2c74-45b7-bd6e-7dfa5c174690 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.255390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.255848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:14.256025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.256487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.256956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] PATH_INFO: `/v3/users/305294dd09944e5088157ede91a30e25` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.257080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.258782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=305294dd09944e5088157ede91a30e25)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.262501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.269255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 983/2002] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:14.269255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.269255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.310139 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:14.326805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305294dd09944e5088157ede91a30e25'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.340584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305294dd09944e5088157ede91a30e25'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.346499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbdcac-f0e7-4426-a253-37d9399c79bc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.351900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5765d3-a8c3-4dca-9f9e-c3240febd595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.352890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5765d3-a8c3-4dca-9f9e-c3240febd595 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.353274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5765d3-a8c3-4dca-9f9e-c3240febd595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.353720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5765d3-a8c3-4dca-9f9e-c3240febd595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.360154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305294dd09944e5088157ede91a30e25'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.361193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305294dd09944e5088157ede91a30e25'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.370586 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5765d3-a8c3-4dca-9f9e-c3240febd595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.372388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305294dd09944e5088157ede91a30e25'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:14.372710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5765d3-a8c3-4dca-9f9e-c3240febd595 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.379872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1020/2003] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:14 2022] DELETE /identity/v3/users/305294dd09944e5088157ede91a30e25 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:14.380662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.382503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.386815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 984/2004] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:14.388569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.389019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.389961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.416572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:14.419062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.515221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7066f-e877-424c-8362-237d66976baa admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.526096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.527179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.528045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.528699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.530241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:14.530868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.532089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 985/2005] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:14.534952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.552469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.554219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.573564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1021/2006] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:14.576642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.612299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e802063-d6f2-47bb-abc0-34dfea2479d9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.619453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.621185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.621502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.621787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.623175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.625974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.661670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee90c8-9461-41f4-ac9f-c8b0673c8c8a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.668268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.668939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.669119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1022/2007] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:14 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:14.669509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.669847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.673274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.682067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.684362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.692770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:14.699105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 986/2008] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:14.746969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750b32-caf9-45ea-b0cc-9c25176006e1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.752057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.752669 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.752949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.753251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.753889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.755821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.783586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1023/2009] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:14 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:14.785151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.796035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550ed1-182e-423d-a96d-000645ad143c service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.801608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.802350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.802699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.803045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.803926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.805650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.816151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 987/2010] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:14.820356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.830547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b9cc-65ae-4c6a-955b-83c67fdc8ff9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.836059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e353b8-91ba-4f11-ae15-e20d57b9ab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.836672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e353b8-91ba-4f11-ae15-e20d57b9ab30 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.836954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e353b8-91ba-4f11-ae15-e20d57b9ab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.837281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e353b8-91ba-4f11-ae15-e20d57b9ab30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.837977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e353b8-91ba-4f11-ae15-e20d57b9ab30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.840040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e353b8-91ba-4f11-ae15-e20d57b9ab30 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.850412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1024/2011] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:14.858813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.871983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb394d4-d0a2-49f9-a25b-d79b80bc4dd4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:14.878504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:14.879415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.879921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.880325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] PATH_INFO: `/v3/projects/b433f2e8c93242ff8f33fa1f87a02e41/users/65f5e9c2b0d64432ae66eaea8bfd20f2/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.899528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b433f2e8c93242ff8f33fa1f87a02e41, user_id=65f5e9c2b0d64432ae66eaea8bfd20f2, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:14.903142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:14.918208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 988/2012] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:14 2022] PUT /identity/v3/projects/b433f2e8c93242ff8f33fa1f87a02e41/users/65f5e9c2b0d64432ae66eaea8bfd20f2/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:14.929217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4f0e-bdae-4959-b2df-6248835ec646 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:14.929509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4f0e-bdae-4959-b2df-6248835ec646 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:14.929918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4f0e-bdae-4959-b2df-6248835ec646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:14.933341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:14.935172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:14.939941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:14.966367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-371a4f0e-bdae-4959-b2df-6248835ec646 None None] MFA Rules not processed for user `65f5e9c2b0d64432ae66eaea8bfd20f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:14.968540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-371a4f0e-bdae-4959-b2df-6248835ec646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.024934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce8ee8-864e-4601-8a56-e547c09adac9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.031318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.042896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.043286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.043567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] PATH_INFO: `/v3/projects/d2c488bf6f064a2f9e4796b0cbef0f19` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.046243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c488bf6f064a2f9e4796b0cbef0f19)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.048554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.065305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1025/2013] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:15.069475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4f0e-bdae-4959-b2df-6248835ec646 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.084005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4f0e-bdae-4959-b2df-6248835ec646 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.093992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.094674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.094957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.095285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.095967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.097950 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.132845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c488bf6f064a2f9e4796b0cbef0f19', 'request_id': 'req-3a5dc5f2-c941-42d7-b294-94705dc1890c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.146250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c488bf6f064a2f9e4796b0cbef0f19', 'request_id': 'req-3a5dc5f2-c941-42d7-b294-94705dc1890c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.152915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1026/2014] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:15 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:15.154058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.155496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c488bf6f064a2f9e4796b0cbef0f19', 'request_id': 'req-3a5dc5f2-c941-42d7-b294-94705dc1890c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.183396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] The token cache is being invalidate because project d2c488bf6f064a2f9e4796b0cbef0f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:15.183748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.185143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a5dc5f2-c941-42d7-b294-94705dc1890c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.187705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 989/2015] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:14 2022] DELETE /identity/v3/projects/d2c488bf6f064a2f9e4796b0cbef0f19 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:15.194050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.194338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.194612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.252593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:15.254246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.271747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51388c-36cb-4764-a3aa-5aee50b4d1d6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.278271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.278878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.279157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.279428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.280057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.282183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.306977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1027/2016] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:15 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:15.308218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.310641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.318573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:15.329112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074ba60-fd59-4d3e-a413-78f22f69b43d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.334746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.335336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 990/2017] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:15.336064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.337486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.338354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.339638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.340113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] PATH_INFO: `/v3/projects/b8d03d510c7846f3bedc7b6a86aea819` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.341563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d03d510c7846f3bedc7b6a86aea819)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.344634 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.355185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b72b9-92c2-4584-90ec-4e87ba4630c6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.359964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ffc99-9d47-4be9-9ca9-cf335623b8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.360572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ffc99-9d47-4be9-9ca9-cf335623b8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.360848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ffc99-9d47-4be9-9ca9-cf335623b8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.361117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ffc99-9d47-4be9-9ca9-cf335623b8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.361783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ffc99-9d47-4be9-9ca9-cf335623b8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.363388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ffc99-9d47-4be9-9ca9-cf335623b8a1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.374108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 991/2018] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:15.375126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.375404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.375675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.397877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:15.399065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.402292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d03d510c7846f3bedc7b6a86aea819', 'request_id': 'req-75617239-1fab-4304-847d-33c0c734c0fb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.409531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d03d510c7846f3bedc7b6a86aea819', 'request_id': 'req-75617239-1fab-4304-847d-33c0c734c0fb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.411515 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:15.416528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d03d510c7846f3bedc7b6a86aea819', 'request_id': 'req-75617239-1fab-4304-847d-33c0c734c0fb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.430157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] The token cache is being invalidate because project b8d03d510c7846f3bedc7b6a86aea819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:15.430468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.431164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.433616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1028/2019] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:15 2022] DELETE /identity/v3/projects/b8d03d510c7846f3bedc7b6a86aea819 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:15.435367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.437593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.464478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 992/2020] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:15.464478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.464478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.475936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0abf9-2354-4456-bb79-5bd9fef3ed2f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.480646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.481376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.481660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.481929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] PATH_INFO: `/v3/users/b01d803e0e774637b60a02eeb2cf57e2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.483107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01d803e0e774637b60a02eeb2cf57e2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.485127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.515496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75617239-1fab-4304-847d-33c0c734c0fb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.522574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.523168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.523392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.523671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] PATH_INFO: `/v3/projects/4d720742ede74b5c8e29d8c5f5ad45bb/users/bc0c2dd641ac4846842aad4036cbfdba/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.546212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d720742ede74b5c8e29d8c5f5ad45bb, user_id=bc0c2dd641ac4846842aad4036cbfdba, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.549853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.562429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01d803e0e774637b60a02eeb2cf57e2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.567494 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1029/2021] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:15 2022] PUT /identity/v3/projects/4d720742ede74b5c8e29d8c5f5ad45bb/users/bc0c2dd641ac4846842aad4036cbfdba/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:15.568696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.580510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01d803e0e774637b60a02eeb2cf57e2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.589959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01d803e0e774637b60a02eeb2cf57e2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.602961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01d803e0e774637b60a02eeb2cf57e2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.603970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01d803e0e774637b60a02eeb2cf57e2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:15.613917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 993/2022] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:15 2022] DELETE /identity/v3/users/b01d803e0e774637b60a02eeb2cf57e2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:15.615094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.616737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.651748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a3bb-31a2-49e4-9d21-24b645631ba7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.711257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:15.716533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498870e-736a-4b6c-a390-ec10f4fd6422 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.726109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.727551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.728131 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.729090 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.730078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.732336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.752670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 994/2023] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:15.753823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.769470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f062733-317e-454a-bbfe-9842e133800e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.776839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40779f8c-9d11-40ac-bb52-39f06ce23b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.777737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779f8c-9d11-40ac-bb52-39f06ce23b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.778121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779f8c-9d11-40ac-bb52-39f06ce23b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.778522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779f8c-9d11-40ac-bb52-39f06ce23b9d admin admin] PATH_INFO: `/v3/projects/4d720742ede74b5c8e29d8c5f5ad45bb/users/bc0c2dd641ac4846842aad4036cbfdba/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.781057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40779f8c-9d11-40ac-bb52-39f06ce23b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d720742ede74b5c8e29d8c5f5ad45bb, user_id=bc0c2dd641ac4846842aad4036cbfdba, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.785287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40779f8c-9d11-40ac-bb52-39f06ce23b9d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.790918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.791562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.794227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.795187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.795894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:15.796205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.797945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:15.801450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 995/2024] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:15 2022] PUT /identity/v3/projects/4d720742ede74b5c8e29d8c5f5ad45bb/users/bc0c2dd641ac4846842aad4036cbfdba/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:15.812307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.812622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.812890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.834608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1030/2025] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:15.837706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:15.841100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.868505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1dc8f-3718-402e-8a8d-33f88f2a15bd admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:15.874629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:15.883798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] MFA Rules not processed for user `bc0c2dd641ac4846842aad4036cbfdba`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:15.887364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:15.948794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:15.949582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:15.949916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:15.950252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:15.952046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:15.954002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.000959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 996/2026] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:16.015695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.019497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:16.037282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1031/2027] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:15 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:16.043778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.149013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4286da-6b08-4ba6-afbe-e1d790a85845 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.178878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8261b-5945-42a1-a384-e65520fe49dc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.184172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.184999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.185263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.186322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.186980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.189048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.226096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1032/2028] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:16.226096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.226484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:16.228253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.228869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.229143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.230243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.232195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:16.242325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b088bdf-b772-4541-b28b-d957221c329c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.248396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.249110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.249490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.249868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] PATH_INFO: `/v3/projects/1a5981630fbb4246a4df7dfe16b0cdd2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.251191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5981630fbb4246a4df7dfe16b0cdd2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.253853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.274295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:16.317938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.319844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.331392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5981630fbb4246a4df7dfe16b0cdd2', 'request_id': 'req-a881bb97-c90a-4154-ae22-6cd44da604b8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:16.335966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 997/2029] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:16.336742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.348793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5981630fbb4246a4df7dfe16b0cdd2', 'request_id': 'req-a881bb97-c90a-4154-ae22-6cd44da604b8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:16.368833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92771d33-2e33-4557-81c9-0b1ee748c4a6 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.375938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5981630fbb4246a4df7dfe16b0cdd2', 'request_id': 'req-a881bb97-c90a-4154-ae22-6cd44da604b8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:16.412203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] The token cache is being invalidate because project 1a5981630fbb4246a4df7dfe16b0cdd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:16.412334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:16.414522 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a881bb97-c90a-4154-ae22-6cd44da604b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:16.418337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1033/2030] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:16 2022] DELETE /identity/v3/projects/1a5981630fbb4246a4df7dfe16b0cdd2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:16.424524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718ef64-d4e9-487b-8e97-815c0b2ab665 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.424847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718ef64-d4e9-487b-8e97-815c0b2ab665 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.425065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718ef64-d4e9-487b-8e97-815c0b2ab665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.456698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:16.486358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8718ef64-d4e9-487b-8e97-815c0b2ab665 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:16.488158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8718ef64-d4e9-487b-8e97-815c0b2ab665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:16.514557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.515186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.515465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.515737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.516381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.518751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.532516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 998/2031] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:16.536338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.538215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:16.554493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597daa4-9f8d-4b9b-a411-da0a694b03c4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.565399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.566793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.567071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.567329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.568027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.570116 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.579710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1034/2032] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:16.591112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf871879-380a-4afc-958e-dd99c3d61a18 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.591499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf871879-380a-4afc-958e-dd99c3d61a18 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.591780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf871879-380a-4afc-958e-dd99c3d61a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.593097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 999/2033] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:16.596955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.609705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef62e39-d3ff-4e74-8bf6-3e3ed01d5fa9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.615608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da3420-c5e7-4a1f-9d1b-337513e65cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.616551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da3420-c5e7-4a1f-9d1b-337513e65cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.616801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da3420-c5e7-4a1f-9d1b-337513e65cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.617170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da3420-c5e7-4a1f-9d1b-337513e65cc5 admin admin] PATH_INFO: `/v3/projects/521940d9fe664526928616f10187d93c/users/767badcf95c04301ba695896a537e2bc/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.620916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bf871879-380a-4afc-958e-dd99c3d61a18 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:16.622698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bf871879-380a-4afc-958e-dd99c3d61a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:16.646450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da3420-c5e7-4a1f-9d1b-337513e65cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=521940d9fe664526928616f10187d93c, user_id=767badcf95c04301ba695896a537e2bc, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.647600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1035/2034] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:16.649726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da3420-c5e7-4a1f-9d1b-337513e65cc5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.652635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf871879-380a-4afc-958e-dd99c3d61a18 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.664344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf871879-380a-4afc-958e-dd99c3d61a18 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.672743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1000/2035] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:16 2022] PUT /identity/v3/projects/521940d9fe664526928616f10187d93c/users/767badcf95c04301ba695896a537e2bc/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:16.684184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9bbc-21f1-4d66-9b76-d82236f47b4f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.685337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9bbc-21f1-4d66-9b76-d82236f47b4f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.686023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9bbc-21f1-4d66-9b76-d82236f47b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.728490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-037b9bbc-21f1-4d66-9b76-d82236f47b4f None None] MFA Rules not processed for user `767badcf95c04301ba695896a537e2bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:16.729901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-037b9bbc-21f1-4d66-9b76-d82236f47b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:16.733234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.733857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.734140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.734411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.735115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:16.736164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.738000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.759945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1036/2036] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:16.764752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.779714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253e4d1-f76f-4b8c-89bf-3c430dd67e20 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.788824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.789960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.790309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.790683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.791791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.794463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.842761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1001/2037] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:16.847127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9bbc-21f1-4d66-9b76-d82236f47b4f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.851691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1037/2038] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:16.854974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.864816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9bbc-21f1-4d66-9b76-d82236f47b4f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.896685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:16.929686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37046da-4e08-4b51-b594-c6073e959814 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:16.937283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.938190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.938619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.938751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.939390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.941883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.943355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:16.943973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:16.944259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:16.944540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:16.945258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:16.947182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:16.968481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1038/2039] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:16.973019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.977916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:16.978739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1002/2040] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:16.982837 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:16.984607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e598541-5bf4-43f8-b5a1-32709436fea9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.047744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421189fd-b42c-4d08-beb1-45971269ee46 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.048592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.049216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.049508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.049775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.050439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.052022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.052479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.053084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.053394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.053666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.054290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.056173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.062447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1039/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:17.066368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.077786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab462-6e39-4d09-93ee-cd9ea2fa4409 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.081356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1003/2042] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:16 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:17.082495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.083082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.083359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.083760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.084619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.085501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.086215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.095578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32b86e-b81e-4c6b-a2aa-827188891f7f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.096102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1040/2043] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:17.100385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.103017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.103625 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.103902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.104170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.104835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.106446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.110379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7fb77-1607-4b0f-a22e-7a93e3da75ed admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.115182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4e50e-f52b-47f9-ac8a-8bb35f643e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.115809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4e50e-f52b-47f9-ac8a-8bb35f643e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.116084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4e50e-f52b-47f9-ac8a-8bb35f643e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.116355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4e50e-f52b-47f9-ac8a-8bb35f643e08 admin admin] PATH_INFO: `/v3/projects/464d0dc91c2b476e8d33648d47ffd2cf/users/7fd1adce68b4491197a635abe7da5b9b/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.117215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1004/2044] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:17.121885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.131538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ac85-a721-4812-b528-4e558b68e9cb admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.133704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4e50e-f52b-47f9-ac8a-8bb35f643e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=464d0dc91c2b476e8d33648d47ffd2cf, user_id=7fd1adce68b4491197a635abe7da5b9b, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.137401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.137750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4e50e-f52b-47f9-ac8a-8bb35f643e08 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.138011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.138295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.138562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.139219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.140858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.152103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1005/2045] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:17.152647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1041/2046] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:17 2022] PUT /identity/v3/projects/464d0dc91c2b476e8d33648d47ffd2cf/users/7fd1adce68b4491197a635abe7da5b9b/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:17.156193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.165033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.165395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.165694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.166959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53954e25-5866-46ba-9e36-327a5f460ace admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.201742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] MFA Rules not processed for user `7fd1adce68b4491197a635abe7da5b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:17.202954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:17.234177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7baf82-6814-45b1-9047-44ceda4748c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.234872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7baf82-6814-45b1-9047-44ceda4748c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.235149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7baf82-6814-45b1-9047-44ceda4748c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.235418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7baf82-6814-45b1-9047-44ceda4748c7 admin admin] PATH_INFO: `/v3/projects/964f8bd92168427792a9d4eef2848396/users/c099641cb5d849baaac2093b27e68a87/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.250469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7baf82-6814-45b1-9047-44ceda4748c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=964f8bd92168427792a9d4eef2848396, user_id=c099641cb5d849baaac2093b27e68a87, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.253112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7baf82-6814-45b1-9047-44ceda4748c7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.268492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1006/2047] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:17 2022] PUT /identity/v3/projects/964f8bd92168427792a9d4eef2848396/users/c099641cb5d849baaac2093b27e68a87/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:17.279453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58528c1b-4ba1-4941-ace0-9d2e9c5e2c87 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.279741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58528c1b-4ba1-4941-ace0-9d2e9c5e2c87 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.280022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58528c1b-4ba1-4941-ace0-9d2e9c5e2c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.287594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:17.297590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1042/2048] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:17.309672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.311144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:17.321240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-58528c1b-4ba1-4941-ace0-9d2e9c5e2c87 None None] MFA Rules not processed for user `c099641cb5d849baaac2093b27e68a87`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:17.322133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-58528c1b-4ba1-4941-ace0-9d2e9c5e2c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:17.377135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8763a-b2ee-48af-b6bf-eb6a720ec19c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.383230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:17.395623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1007/2049] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:17.401489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58528c1b-4ba1-4941-ace0-9d2e9c5e2c87 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.444088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.444723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.445004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.445461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.475209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58528c1b-4ba1-4941-ace0-9d2e9c5e2c87 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.481441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.482065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.482346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.482617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.483256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.485294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.527397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1008/2050] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:17 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:17.528102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.529594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.531948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.550255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1043/2051] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:17.574482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.590059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fbe7f-e8ae-4cd5-bfe1-66b76f91b2e8 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.630088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e871b6-ea33-410c-ab25-b7f7ffac3534 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.636541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.637166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.638662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.638845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.639458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.641851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.681292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1009/2052] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:17 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:17.684321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.696799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.697517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.697795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.699831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] PATH_INFO: `/v3/users/7fd1adce68b4491197a635abe7da5b9b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.699831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd1adce68b4491197a635abe7da5b9b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.701804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.703924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff97f8-abf6-421e-a397-64b7e95f90a9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.711681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.712678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.713137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.714145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.714976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.717238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.718950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:17.740732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1010/2053] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:17.743062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.756317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c365c6-2598-495a-a009-95a717815b85 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.761398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.762073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.762349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.762617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] PATH_INFO: `/v3/projects/9987ef7f183547aa8e35ee06962fd8e9/users/f25790f8c6744ed1b887fba7fc951203/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.771192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd1adce68b4491197a635abe7da5b9b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:17.779639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd1adce68b4491197a635abe7da5b9b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:17.780654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9987ef7f183547aa8e35ee06962fd8e9, user_id=f25790f8c6744ed1b887fba7fc951203, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.783642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:17.788380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:17.798164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd1adce68b4491197a635abe7da5b9b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:17.798918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd1adce68b4491197a635abe7da5b9b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:17.799046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1011/2054] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:17 2022] PUT /identity/v3/projects/9987ef7f183547aa8e35ee06962fd8e9/users/f25790f8c6744ed1b887fba7fc951203/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:17.802808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.807717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce9694d9-f440-4c25-8c4d-bf693a207aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd1adce68b4491197a635abe7da5b9b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:17.814458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1044/2055] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:17 2022] DELETE /identity/v3/users/7fd1adce68b4491197a635abe7da5b9b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:17.818762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.819221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.819566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.841820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:17.843218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:17.896881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f24a7d-8966-4c9d-bcc8-f42a31c9ba99 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:17.926731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1045/2056] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:17.937213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:17.939742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:17.985959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:17.987025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:17.987376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:17.987892 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:17.989102 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:17.991346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:18.015714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1012/2057] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:18.020967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:18.023773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:18.023973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb27c2c-f3d0-401b-b8e1-7fd11f177c1a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:18.032218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89c34f-1c82-4414-a429-393cd7818b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:18.032951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89c34f-1c82-4414-a429-393cd7818b43 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:18.033342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89c34f-1c82-4414-a429-393cd7818b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:18.033617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89c34f-1c82-4414-a429-393cd7818b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:18.044872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b163cf5-a18f-4f88-a6b3-c3de74a34f91 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:18.046474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89c34f-1c82-4414-a429-393cd7818b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:18.049019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89c34f-1c82-4414-a429-393cd7818b43 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:18.052222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:18.053157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:18.053577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:18.054058 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] PATH_INFO: `/v3/projects/9987ef7f183547aa8e35ee06962fd8e9/users/f25790f8c6744ed1b887fba7fc951203/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:18.058631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9987ef7f183547aa8e35ee06962fd8e9, user_id=f25790f8c6744ed1b887fba7fc951203, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:18.062717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:18.063306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1046/2058] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:18.090523 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1013/2059] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:18 2022] PUT /identity/v3/projects/9987ef7f183547aa8e35ee06962fd8e9/users/f25790f8c6744ed1b887fba7fc951203/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:18.092417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:18.092729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:18.093112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:18.137753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] MFA Rules not processed for user `f25790f8c6744ed1b887fba7fc951203`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:18.140111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:18.242593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1047/2060] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:18.258188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:18.260124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:18.331168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912712a7-ca6c-4b2e-a3d0-ade2c734ef33 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:18.335414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12b4a9-27b1-47e0-93ec-3a5433d5a6ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:18.336036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b4a9-27b1-47e0-93ec-3a5433d5a6ae service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:18.336318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b4a9-27b1-47e0-93ec-3a5433d5a6ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:18.336591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b4a9-27b1-47e0-93ec-3a5433d5a6ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:18.338391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ef12b4a9-27b1-47e0-93ec-3a5433d5a6ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:18.360047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:18.411540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12b4a9-27b1-47e0-93ec-3a5433d5a6ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:18.413868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12b4a9-27b1-47e0-93ec-3a5433d5a6ae service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:18.423454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1014/2061] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:18.627186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:18.628640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:18.647744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:18.767153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3fc5a-df85-4ae1-b94a-6f83b40ec0a7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:18.774576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:18.774576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:18.774858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:18.775230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] PATH_INFO: `/v3/projects/464d0dc91c2b476e8d33648d47ffd2cf` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:18.776838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=464d0dc91c2b476e8d33648d47ffd2cf)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:18.779696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:18.848293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '464d0dc91c2b476e8d33648d47ffd2cf', 'request_id': 'req-46b5d227-356c-4ba5-83e8-75905be0c117'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:18.864416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '464d0dc91c2b476e8d33648d47ffd2cf', 'request_id': 'req-46b5d227-356c-4ba5-83e8-75905be0c117'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:18.876346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '464d0dc91c2b476e8d33648d47ffd2cf', 'request_id': 'req-46b5d227-356c-4ba5-83e8-75905be0c117'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:18.894277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] The token cache is being invalidate because project 464d0dc91c2b476e8d33648d47ffd2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:18.894657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:18.895248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b5d227-356c-4ba5-83e8-75905be0c117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:18.897157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1048/2062] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:18 2022] DELETE /identity/v3/projects/464d0dc91c2b476e8d33648d47ffd2cf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:18.902734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41642-7496-4bf1-962e-418655591357 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:18.903280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41642-7496-4bf1-962e-418655591357 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:18.903916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41642-7496-4bf1-962e-418655591357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:18.930469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-42c41642-7496-4bf1-962e-418655591357 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:18.932780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-42c41642-7496-4bf1-962e-418655591357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.013929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1015/2063] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:19.024160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622b414-3537-43a5-b794-7fc258ef269f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.025717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622b414-3537-43a5-b794-7fc258ef269f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.026887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622b414-3537-43a5-b794-7fc258ef269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.047126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c622b414-3537-43a5-b794-7fc258ef269f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:19.048441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c622b414-3537-43a5-b794-7fc258ef269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.058400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1049/2064] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:19.062317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41642-7496-4bf1-962e-418655591357 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.074278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41642-7496-4bf1-962e-418655591357 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.080428 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.081045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.082075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.082075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.082757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:19.083063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.085748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.104908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1016/2065] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:19.109492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622b414-3537-43a5-b794-7fc258ef269f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.121856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622b414-3537-43a5-b794-7fc258ef269f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.128647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.129296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.131293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.131293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.131293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.144140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.158820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.160787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.175757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab94e80-8f86-4a70-be23-ad00a97a7871 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.183557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.184352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.184635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.184934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] PATH_INFO: `/v3/users/65f5e9c2b0d64432ae66eaea8bfd20f2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.186132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f5e9c2b0d64432ae66eaea8bfd20f2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.188255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.188390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1050/2066] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:19 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:19.189340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.270383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f5e9c2b0d64432ae66eaea8bfd20f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:19.285961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f5e9c2b0d64432ae66eaea8bfd20f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:19.296242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f5e9c2b0d64432ae66eaea8bfd20f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:19.308188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f5e9c2b0d64432ae66eaea8bfd20f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:19.309226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c24a87f5-5d00-45b8-82c2-fdef22c1d701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f5e9c2b0d64432ae66eaea8bfd20f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:19.317352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d123-8ddc-4463-8791-676d2f5b6314 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.323693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1017/2067] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:19 2022] DELETE /identity/v3/users/65f5e9c2b0d64432ae66eaea8bfd20f2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:19.325248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.325538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.325822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.347859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:19.359856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:19.361099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.374785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:19.430325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.431030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.431385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.431660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.433022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.435347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.465833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1018/2068] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:19.470568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1051/2069] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:19 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:19.474672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.476099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.484763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.486759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.493533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a623-e995-4ad2-960c-4ec9dbedc19e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.505233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.505877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.506157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.506424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.507068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.509012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.528111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1019/2070] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:19.529388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.544115 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9c101-405e-4f37-b6d4-7876b722263b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.551810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.552517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.552784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.554401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.554401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.555388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.573248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1020/2071] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:19.576332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.585596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc817cb-3fb4-4d49-bf63-e7582138c6d1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.587420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322eba2-4d5d-4997-b2bc-1f48e5ed3c9f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.589485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517396df-738d-4161-a4aa-a24ad0c73084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.590100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517396df-738d-4161-a4aa-a24ad0c73084 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.590393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517396df-738d-4161-a4aa-a24ad0c73084 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.590645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517396df-738d-4161-a4aa-a24ad0c73084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.596028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.597280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.597574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.598154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] PATH_INFO: `/v3/projects/fab43244f2a043f988668489b6b7f6c4/users/e4dedbd8d7724d218de94ff032833582/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.605652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517396df-738d-4161-a4aa-a24ad0c73084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.607363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517396df-738d-4161-a4aa-a24ad0c73084 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.617541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab43244f2a043f988668489b6b7f6c4, user_id=e4dedbd8d7724d218de94ff032833582, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.620831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.633036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1052/2072] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:19.652205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1021/2073] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:19 2022] PUT /identity/v3/projects/fab43244f2a043f988668489b6b7f6c4/users/e4dedbd8d7724d218de94ff032833582/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:19.655165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.655165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.655165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.686903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] MFA Rules not processed for user `e4dedbd8d7724d218de94ff032833582`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:19.688246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.801551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1053/2074] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:19.809958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:19.812337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:19.895336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2982ae5-87da-4070-a40f-6e4031d47884 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:19.900798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:19.902156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:19.902438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:19.902718 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:19.915315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:19.917649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:19.930747 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1022/2075] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:20.333422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:20.333422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:20.360870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7acc2-3562-493a-8b63-0bb8c379766e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:20.368677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:20.369358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:20.369679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:20.370080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:20.376953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:20.475735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:20.477522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:20.553632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:20.603736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:20.605316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:20.614453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f1041-d701-474b-a495-0f4e9d0496ac service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:20.623313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:20.624830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:20.625127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1054/2076] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:20.626346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:20.626960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] PATH_INFO: `/v3/projects/b433f2e8c93242ff8f33fa1f87a02e41` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:20.628914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b433f2e8c93242ff8f33fa1f87a02e41)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:20.631369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:20.660855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:20.723566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b433f2e8c93242ff8f33fa1f87a02e41', 'request_id': 'req-afb3022c-54df-4fdc-a3e6-096049d07a0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.731670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b433f2e8c93242ff8f33fa1f87a02e41', 'request_id': 'req-afb3022c-54df-4fdc-a3e6-096049d07a0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.738820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b433f2e8c93242ff8f33fa1f87a02e41', 'request_id': 'req-afb3022c-54df-4fdc-a3e6-096049d07a0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.757535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] The token cache is being invalidate because project b433f2e8c93242ff8f33fa1f87a02e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:20.758301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.759058 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afb3022c-54df-4fdc-a3e6-096049d07a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.766702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1023/2077] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:20 2022] DELETE /identity/v3/projects/b433f2e8c93242ff8f33fa1f87a02e41 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:20.767807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:20.786720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099525f-a13f-413e-b1d4-d02a99bd213a service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:20.870843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:20.871650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:20.871923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:20.872227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] PATH_INFO: `/v3/users/bc0c2dd641ac4846842aad4036cbfdba` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:20.873462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc0c2dd641ac4846842aad4036cbfdba)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:20.875424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:20.927924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc0c2dd641ac4846842aad4036cbfdba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.936098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc0c2dd641ac4846842aad4036cbfdba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.950685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc0c2dd641ac4846842aad4036cbfdba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.951349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc0c2dd641ac4846842aad4036cbfdba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.959790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc0c2dd641ac4846842aad4036cbfdba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:20.967303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1055/2078] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:20 2022] DELETE /identity/v3/users/bc0c2dd641ac4846842aad4036cbfdba => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:21.215099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:21.215386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:21.215684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:21.248417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] MFA Rules not processed for user `ca360b149ac8402282733edfc4c67f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:21.249692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:21.326973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:21.328496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:21.380628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1024/2079] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:21.381544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:21.383786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:21.446750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d005e-b500-4259-86e5-2bff2182b42b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:21.452723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa896e26-7b8f-4b97-9059-9a6f9e11e456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:21.453503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa896e26-7b8f-4b97-9059-9a6f9e11e456 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:21.453881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa896e26-7b8f-4b97-9059-9a6f9e11e456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:21.454257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa896e26-7b8f-4b97-9059-9a6f9e11e456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:21.456643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fa896e26-7b8f-4b97-9059-9a6f9e11e456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:21.473670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:21.522462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89011dda-d8dc-4d02-b95c-2236825c5ddb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:21.530663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:21.531435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:21.531812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:21.532148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] PATH_INFO: `/v3/projects/4d720742ede74b5c8e29d8c5f5ad45bb` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:21.533380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d720742ede74b5c8e29d8c5f5ad45bb)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:21.536568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:21.570668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa896e26-7b8f-4b97-9059-9a6f9e11e456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:21.572239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa896e26-7b8f-4b97-9059-9a6f9e11e456 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:21.585290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1056/2080] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:21.614244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d720742ede74b5c8e29d8c5f5ad45bb', 'request_id': 'req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:21.616623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:21.637258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d720742ede74b5c8e29d8c5f5ad45bb', 'request_id': 'req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:21.672698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d720742ede74b5c8e29d8c5f5ad45bb', 'request_id': 'req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:21.727006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] The token cache is being invalidate because project 4d720742ede74b5c8e29d8c5f5ad45bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:21.730305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:21.732629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-77c3b5d5-dfc8-4b5f-92b0-25ae29974301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:21.743471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1025/2081] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:21 2022] DELETE /identity/v3/projects/4d720742ede74b5c8e29d8c5f5ad45bb => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:21.750996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e02f1e-934e-43a3-ad1d-866d7e11a04b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:21.751287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e02f1e-934e-43a3-ad1d-866d7e11a04b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:21.751567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e02f1e-934e-43a3-ad1d-866d7e11a04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:21.778870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-93e02f1e-934e-43a3-ad1d-866d7e11a04b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:21.780283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-93e02f1e-934e-43a3-ad1d-866d7e11a04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:21.859219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1057/2082] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:21.867312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0019c-7250-4615-bc85-12a4e7ff81ac None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:21.867578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0019c-7250-4615-bc85-12a4e7ff81ac None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:21.867849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0019c-7250-4615-bc85-12a4e7ff81ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:21.920674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cad0019c-7250-4615-bc85-12a4e7ff81ac None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:21.923677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cad0019c-7250-4615-bc85-12a4e7ff81ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:21.946007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1026/2083] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:21.949601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e02f1e-934e-43a3-ad1d-866d7e11a04b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:21.964383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e02f1e-934e-43a3-ad1d-866d7e11a04b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:21.971224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:21.972232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:21.972759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:21.973246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:21.974261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:21.974594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:21.978891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:22.039665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1058/2084] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:22.046229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0019c-7250-4615-bc85-12a4e7ff81ac None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:22.066365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0019c-7250-4615-bc85-12a4e7ff81ac None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:22.089119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:22.089791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.090079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.090379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.091059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:22.096771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:22.145756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1027/2085] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:22 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:22.150124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:22.171579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2900c-bbb4-4a42-b3a7-5cbc6b891070 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:22.305477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:22.306096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.306374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.306647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.307320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:22.309531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:22.358825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1059/2086] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:22 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:22.362495 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:22.374548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36673e80-9854-4c99-a5ef-ef1c4cf1062d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:22.380399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:22.382183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.382484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.382764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.383494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:22.385104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:22.399151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1028/2087] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:22.404577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:22.424259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f339e4-af06-43c4-9435-b3d375699d5c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:22.429702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6be09-110a-4412-9b98-4b8dc3e05e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:22.430323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6be09-110a-4412-9b98-4b8dc3e05e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.430598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6be09-110a-4412-9b98-4b8dc3e05e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.430865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6be09-110a-4412-9b98-4b8dc3e05e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.431695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6be09-110a-4412-9b98-4b8dc3e05e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:22.433533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6be09-110a-4412-9b98-4b8dc3e05e71 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:22.451630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1060/2088] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:22.453747 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:22.465390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68934765-bbe2-4fe5-9108-4312516bab00 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:22.483104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:22.483878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.484169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.484446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] PATH_INFO: `/v3/projects/090e4bcba84f427d8c8aa7d1d11fef56/users/36f54af102e14a1294833bb37d36348a/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.520292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090e4bcba84f427d8c8aa7d1d11fef56, user_id=36f54af102e14a1294833bb37d36348a, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:22.525472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:22.556105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1029/2089] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:22 2022] PUT /identity/v3/projects/090e4bcba84f427d8c8aa7d1d11fef56/users/36f54af102e14a1294833bb37d36348a/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:22.568204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.568493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.568802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.616048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] MFA Rules not processed for user `36f54af102e14a1294833bb37d36348a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:22.617242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:22.622013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:22.700717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1061/2090] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:22.711251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:22.713032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:22.790985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e208c68-6cc1-41fb-bd7e-e255ee095885 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:22.795405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:22.796214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.796612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.797383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.810477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:22.812056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:22.816159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:22.821325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:22.845614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1030/2091] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:22.954656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a225814-37cf-4b61-8f96-7e4e83e13d7b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:22.960934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:22.961704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:22.962058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:22.962454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] PATH_INFO: `/v3/users/1e123765a8724a89863d74221001dcd1` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:22.964212 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e123765a8724a89863d74221001dcd1)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:22.966742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.078732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e123765a8724a89863d74221001dcd1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.086775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e123765a8724a89863d74221001dcd1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.100394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e123765a8724a89863d74221001dcd1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.102096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e123765a8724a89863d74221001dcd1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.128504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05062f17-73ad-4709-8243-b1b045707fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e123765a8724a89863d74221001dcd1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.146565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1062/2092] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:22 2022] DELETE /identity/v3/users/1e123765a8724a89863d74221001dcd1 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:23.369434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:23.383149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78205b5-5645-4168-8315-9f3d156ced64 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:23.442772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:23.443401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.443707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.443978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] PATH_INFO: `/v3/projects/3c2219444c1b4ad0a395880494ddbbe3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.445134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c2219444c1b4ad0a395880494ddbbe3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:23.447066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.468243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:23.493868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c2219444c1b4ad0a395880494ddbbe3', 'request_id': 'req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.500752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c2219444c1b4ad0a395880494ddbbe3', 'request_id': 'req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.506350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c2219444c1b4ad0a395880494ddbbe3', 'request_id': 'req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.519103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] The token cache is being invalidate because project 3c2219444c1b4ad0a395880494ddbbe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:23.519430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.520061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a95a3d2b-84bc-41c9-95c0-ead61edb6c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:23.521577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1031/2093] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:23 2022] DELETE /identity/v3/projects/3c2219444c1b4ad0a395880494ddbbe3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:23.527788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad92833-6ca9-43a2-bb23-b6394c6d4e97 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.528144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad92833-6ca9-43a2-bb23-b6394c6d4e97 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.528486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad92833-6ca9-43a2-bb23-b6394c6d4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.548860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0ad92833-6ca9-43a2-bb23-b6394c6d4e97 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:23.550217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ad92833-6ca9-43a2-bb23-b6394c6d4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:23.571679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:23.616419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1063/2094] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:23.624591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90315-0ef9-459d-8c80-632b0327cb4f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.625078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90315-0ef9-459d-8c80-632b0327cb4f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.625167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90315-0ef9-459d-8c80-632b0327cb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.644549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9bf90315-0ef9-459d-8c80-632b0327cb4f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:23.645769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9bf90315-0ef9-459d-8c80-632b0327cb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:23.654268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1032/2095] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:23.658284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad92833-6ca9-43a2-bb23-b6394c6d4e97 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:23.668955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad92833-6ca9-43a2-bb23-b6394c6d4e97 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:23.673736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:23.674328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.674608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.674880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.675524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:23.675827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:23.677456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.688775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1064/2096] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:23.692773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90315-0ef9-459d-8c80-632b0327cb4f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:23.701846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90315-0ef9-459d-8c80-632b0327cb4f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:23.706392 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:23.706973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.707247 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.707515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.708145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:23.710066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.737938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1033/2097] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:23 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:23.741990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:23.751778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d66c2-eb80-4669-bbcd-f0e4e50f65b2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:23.806561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:23.807174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.807485 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.807759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.808423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:23.810393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.831273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1065/2098] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:23 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:23.835344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:23.846019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b1e98-d995-4d05-8d45-202603547e70 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:23.850563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:23.851151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.851425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.851690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.852321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:23.853921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.864051 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1034/2099] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:23.867846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:23.877683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d61e3d-a50d-4091-ac77-ec0300a69b02 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:23.882262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dabe86-363d-4a77-8ffd-a71c78d4eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:23.882841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabe86-363d-4a77-8ffd-a71c78d4eea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.883119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabe86-363d-4a77-8ffd-a71c78d4eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.883390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabe86-363d-4a77-8ffd-a71c78d4eea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.884030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dabe86-363d-4a77-8ffd-a71c78d4eea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:23.885657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dabe86-363d-4a77-8ffd-a71c78d4eea9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.895837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1066/2100] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:23.899746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:23.910316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577b302-6829-4cb1-94cc-e8c5b37a4826 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:23.914994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:23.915635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.915912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.916180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] PATH_INFO: `/v3/projects/e498d32cc51a4bad9cb21e58985ce63e/users/b72887db5ed34dbebe0d5745cf77c4a8/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.930861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e498d32cc51a4bad9cb21e58985ce63e, user_id=b72887db5ed34dbebe0d5745cf77c4a8, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:23.933528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:23.946272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1035/2101] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:23 2022] PUT /identity/v3/projects/e498d32cc51a4bad9cb21e58985ce63e/users/b72887db5ed34dbebe0d5745cf77c4a8/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:23.951089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d261d28-7295-497c-9057-acd19b292dca None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:23.951372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d261d28-7295-497c-9057-acd19b292dca None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:23.951637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d261d28-7295-497c-9057-acd19b292dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:23.970265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2d261d28-7295-497c-9057-acd19b292dca None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:23.971437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2d261d28-7295-497c-9057-acd19b292dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:24.030333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1067/2102] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:24.040396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:24.042228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:24.101233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c98ca-8ba0-4690-bc93-41eecba8ed2c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:24.104898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:24.105540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:24.105824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:24.106096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:24.118311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:24.121124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:24.130078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1036/2103] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:30.002791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:30.002791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:30.002791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:30.035689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] MFA Rules not processed for user `b72887db5ed34dbebe0d5745cf77c4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:30.037028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:30.063711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:30.116892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1068/2104] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:30.130868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:30.132810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:30.241396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:30.286396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87aaf6-bed4-461e-8946-9b47e2cc155a service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:30.293951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2641c9b-8d37-4b8d-95af-fab1f6e5a7ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:30.294694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2641c9b-8d37-4b8d-95af-fab1f6e5a7ce service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:30.295029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2641c9b-8d37-4b8d-95af-fab1f6e5a7ce service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:30.295367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2641c9b-8d37-4b8d-95af-fab1f6e5a7ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:30.305253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2641c9b-8d37-4b8d-95af-fab1f6e5a7ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:30.307577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2641c9b-8d37-4b8d-95af-fab1f6e5a7ce service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:30.318010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1037/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:31.483087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:31.485291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:31.501332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816d6fe-fef8-4a84-a142-57e0af924c57 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:31.507052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:31.507941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:31.508576 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:31.509040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] PATH_INFO: `/v3/users/767badcf95c04301ba695896a537e2bc` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:31.511246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=767badcf95c04301ba695896a537e2bc)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:31.513790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:31.580219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767badcf95c04301ba695896a537e2bc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:31.590386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767badcf95c04301ba695896a537e2bc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:31.613887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767badcf95c04301ba695896a537e2bc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:31.617601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767badcf95c04301ba695896a537e2bc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:31.629423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767badcf95c04301ba695896a537e2bc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:31.653289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1069/2106] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:31 2022] DELETE /identity/v3/users/767badcf95c04301ba695896a537e2bc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:31.657592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ff7fc-9896-486d-aad0-3594bf9afcb9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:31.658149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ff7fc-9896-486d-aad0-3594bf9afcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:31.658149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ff7fc-9896-486d-aad0-3594bf9afcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:31.687472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-544ff7fc-9896-486d-aad0-3594bf9afcb9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:31.689526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-544ff7fc-9896-486d-aad0-3594bf9afcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:31.770290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1038/2107] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:31.783872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:31.786441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:31.853589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f330-99ee-4878-be1c-704e4b6aa871 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:31.860111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:31.860878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:31.861278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:31.861666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:31.893235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:31.894984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:31.906802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1070/2108] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:32.169786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431a014-5cdf-4100-86ca-417e545a2518 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:32.170365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431a014-5cdf-4100-86ca-417e545a2518 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:32.170858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431a014-5cdf-4100-86ca-417e545a2518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:32.194781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0431a014-5cdf-4100-86ca-417e545a2518 None None] MFA Rules not processed for user `b72887db5ed34dbebe0d5745cf77c4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:32.196061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0431a014-5cdf-4100-86ca-417e545a2518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:32.295063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1039/2109] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:32.621746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:32.623143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:32.641055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a62f6d-325e-4eb4-849c-37abc793e936 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:32.649427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:32.652688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:32.653357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:32.654153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] PATH_INFO: `/v3/projects/521940d9fe664526928616f10187d93c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:32.657308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] RBAC: Authorizing `identity:delete_project(project_id=521940d9fe664526928616f10187d93c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:32.675613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:32.784224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:32.807802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521940d9fe664526928616f10187d93c', 'request_id': 'req-a9597de5-95bf-4bb0-94c8-ce9b1494955f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:32.815457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521940d9fe664526928616f10187d93c', 'request_id': 'req-a9597de5-95bf-4bb0-94c8-ce9b1494955f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:32.821689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521940d9fe664526928616f10187d93c', 'request_id': 'req-a9597de5-95bf-4bb0-94c8-ce9b1494955f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:32.839362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] The token cache is being invalidate because project 521940d9fe664526928616f10187d93c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:32.839654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:32.840366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:32.842200 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1071/2110] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:32 2022] DELETE /identity/v3/projects/521940d9fe664526928616f10187d93c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:32.846457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431a014-5cdf-4100-86ca-417e545a2518 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:32.857980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431a014-5cdf-4100-86ca-417e545a2518 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:32.926891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:32.927638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:32.928064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:32.928267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] PATH_INFO: `/v3/users/c099641cb5d849baaac2093b27e68a87` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:32.929648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c099641cb5d849baaac2093b27e68a87)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:32.931577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:32.956155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:32.995508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c099641cb5d849baaac2093b27e68a87'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.023370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c099641cb5d849baaac2093b27e68a87'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.034666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c099641cb5d849baaac2093b27e68a87'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.047300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c099641cb5d849baaac2093b27e68a87'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.047867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c099641cb5d849baaac2093b27e68a87'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.055092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1040/2111] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:32 2022] DELETE /identity/v3/users/c099641cb5d849baaac2093b27e68a87 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:33.472901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:33.474725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:33.491635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9597de5-95bf-4bb0-94c8-ce9b1494955f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:33.554299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:33.555018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:33.555299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:33.555607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] PATH_INFO: `/v3/projects/964f8bd92168427792a9d4eef2848396` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:33.557645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=964f8bd92168427792a9d4eef2848396)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:33.559916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:33.613872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '964f8bd92168427792a9d4eef2848396', 'request_id': 'req-74a69754-2750-4cfb-89bd-da68d902ad12'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.621901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '964f8bd92168427792a9d4eef2848396', 'request_id': 'req-74a69754-2750-4cfb-89bd-da68d902ad12'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.628000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '964f8bd92168427792a9d4eef2848396', 'request_id': 'req-74a69754-2750-4cfb-89bd-da68d902ad12'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.643387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] The token cache is being invalidate because project 964f8bd92168427792a9d4eef2848396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:33.643682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.644744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.646103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1072/2112] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:33 2022] DELETE /identity/v3/projects/964f8bd92168427792a9d4eef2848396 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:33.650306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:33.652394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:33.666301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b210-d3ca-4088-8e51-66aa5b1960a7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:33.741523 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:33.742217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:33.742724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:33.743141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] PATH_INFO: `/v3/users/f25790f8c6744ed1b887fba7fc951203` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:33.744488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25790f8c6744ed1b887fba7fc951203)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:33.747197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:33.799621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25790f8c6744ed1b887fba7fc951203'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.812287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25790f8c6744ed1b887fba7fc951203'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.820341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25790f8c6744ed1b887fba7fc951203'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.828701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25790f8c6744ed1b887fba7fc951203'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.829715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-31d9d836-d000-4190-8b52-f50d565ff188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25790f8c6744ed1b887fba7fc951203'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:33.838643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1041/2113] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:33 2022] DELETE /identity/v3/users/f25790f8c6744ed1b887fba7fc951203 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:34.144442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.145921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:34.162767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69754-2750-4cfb-89bd-da68d902ad12 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.229000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:34.251036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05792869-81d5-4214-883a-74f445979908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.251672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05792869-81d5-4214-883a-74f445979908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.252254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05792869-81d5-4214-883a-74f445979908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.252523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05792869-81d5-4214-883a-74f445979908 admin admin] PATH_INFO: `/v3/projects/9987ef7f183547aa8e35ee06962fd8e9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.253715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05792869-81d5-4214-883a-74f445979908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9987ef7f183547aa8e35ee06962fd8e9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.255840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05792869-81d5-4214-883a-74f445979908 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.306157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05792869-81d5-4214-883a-74f445979908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9987ef7f183547aa8e35ee06962fd8e9', 'request_id': 'req-05792869-81d5-4214-883a-74f445979908'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:34.317658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05792869-81d5-4214-883a-74f445979908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9987ef7f183547aa8e35ee06962fd8e9', 'request_id': 'req-05792869-81d5-4214-883a-74f445979908'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:34.323801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05792869-81d5-4214-883a-74f445979908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9987ef7f183547aa8e35ee06962fd8e9', 'request_id': 'req-05792869-81d5-4214-883a-74f445979908'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:34.338223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05792869-81d5-4214-883a-74f445979908 admin admin] The token cache is being invalidate because project 9987ef7f183547aa8e35ee06962fd8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:34.339261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05792869-81d5-4214-883a-74f445979908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:34.340451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-05792869-81d5-4214-883a-74f445979908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:34.341893 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1073/2114] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:34 2022] DELETE /identity/v3/projects/9987ef7f183547aa8e35ee06962fd8e9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:34.346528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fdc32-70b5-41c6-bb93-a8752d41c898 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.346972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fdc32-70b5-41c6-bb93-a8752d41c898 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.347066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fdc32-70b5-41c6-bb93-a8752d41c898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.367690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c22fdc32-70b5-41c6-bb93-a8752d41c898 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:34.368891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c22fdc32-70b5-41c6-bb93-a8752d41c898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:34.379554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:34.435952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1042/2115] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:34.444053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58312670-b4eb-4794-bb46-fd106cfa84b8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.444346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58312670-b4eb-4794-bb46-fd106cfa84b8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.444658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58312670-b4eb-4794-bb46-fd106cfa84b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.466851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-58312670-b4eb-4794-bb46-fd106cfa84b8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:34.468710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-58312670-b4eb-4794-bb46-fd106cfa84b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:34.479697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1074/2116] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:34.482081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fdc32-70b5-41c6-bb93-a8752d41c898 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.494259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fdc32-70b5-41c6-bb93-a8752d41c898 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.501838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.502458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.502732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.503001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.503648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:34.503946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.505663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.518676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1043/2117] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:34.523375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58312670-b4eb-4794-bb46-fd106cfa84b8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.535306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58312670-b4eb-4794-bb46-fd106cfa84b8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.541293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.541938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.542220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.542493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.543202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.545101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.576861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1075/2118] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:34 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:34.581501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.592018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b80001-74ee-4560-a8db-5877303bb786 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.654648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.655325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.655607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.655916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.656701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.658656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.679864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1044/2119] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:34 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:34.684879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.696201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779357c-b74f-4756-975c-35b2cad81061 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.701709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.702571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.702858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.703182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.704093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.705969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.717564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1076/2120] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:34.721591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.733022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1b15-a67c-486b-bed7-9ea5da747118 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.737757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.738360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.738642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.738949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.739642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.741260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.754204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1045/2121] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:34.767055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.778210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a759-d7c3-4d03-93d7-7fde0d37ed36 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.788204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1457bab-34e7-40c5-9b92-d5b613176964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.789099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1457bab-34e7-40c5-9b92-d5b613176964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.789585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1457bab-34e7-40c5-9b92-d5b613176964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.790099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1457bab-34e7-40c5-9b92-d5b613176964 admin admin] PATH_INFO: `/v3/projects/3f711f25c212427db970f02391c79d1c/users/0eccc82843084cb392b1f15ad2ac9268/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.818199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.820065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1457bab-34e7-40c5-9b92-d5b613176964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f711f25c212427db970f02391c79d1c, user_id=0eccc82843084cb392b1f15ad2ac9268, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.820217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:34.822708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1457bab-34e7-40c5-9b92-d5b613176964 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.836398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de176-c261-4f33-87bc-05ea94794298 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:34.838665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1077/2122] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:34 2022] PUT /identity/v3/projects/3f711f25c212427db970f02391c79d1c/users/0eccc82843084cb392b1f15ad2ac9268/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:34.844310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.844658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.844998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.866698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:34.867931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:34.933138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:34.934842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:34.935179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:34.936027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] PATH_INFO: `/v3/users/e4dedbd8d7724d218de94ff032833582` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:34.943181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4dedbd8d7724d218de94ff032833582)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:34.950875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:34.986524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1078/2123] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:34.991495 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:34.996858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:34.998876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:35.023951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:35.052794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4dedbd8d7724d218de94ff032833582'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:35.071475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4dedbd8d7724d218de94ff032833582'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:35.082216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4dedbd8d7724d218de94ff032833582'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:35.097408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4dedbd8d7724d218de94ff032833582'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:35.098727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aec5d-6538-4ee1-8372-5a3ded699d49 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:35.099640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-17f6925c-76e7-485c-b867-2b955adf2310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4dedbd8d7724d218de94ff032833582'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:35.107888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1046/2124] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:34 2022] DELETE /identity/v3/users/e4dedbd8d7724d218de94ff032833582 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:35.110492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:35.111094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:35.111375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:35.111643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:35.113536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:35.113869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:35.114138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:35.115794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:35.136405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:35.138474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:35.141461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:35.142754 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:35.164479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1079/2125] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:35.234909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1047/2126] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:35.248040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:35.249442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:35.264207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71482-b44c-49a4-b85b-2ebe6635796d service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:35.346380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3b342-5c20-4d44-87c3-519ac532797c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:35.347033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3b342-5c20-4d44-87c3-519ac532797c service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:35.347312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3b342-5c20-4d44-87c3-519ac532797c service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:35.347582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3b342-5c20-4d44-87c3-519ac532797c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:35.365904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3b342-5c20-4d44-87c3-519ac532797c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:35.367898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3b342-5c20-4d44-87c3-519ac532797c service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:35.379471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1080/2127] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:36.031514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:36.033490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:36.049671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7881a1-97d4-401d-8054-a4ee6ef1af02 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:36.056043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:36.056697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.056997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.057834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] PATH_INFO: `/v3/projects/fab43244f2a043f988668489b6b7f6c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.059272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab43244f2a043f988668489b6b7f6c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:36.061585 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:36.167079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab43244f2a043f988668489b6b7f6c4', 'request_id': 'req-cc5de69c-f049-43bc-a243-c16e353d4652'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:36.176793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab43244f2a043f988668489b6b7f6c4', 'request_id': 'req-cc5de69c-f049-43bc-a243-c16e353d4652'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:36.188194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab43244f2a043f988668489b6b7f6c4', 'request_id': 'req-cc5de69c-f049-43bc-a243-c16e353d4652'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:36.212877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] The token cache is being invalidate because project fab43244f2a043f988668489b6b7f6c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:36.213406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:36.214232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5de69c-f049-43bc-a243-c16e353d4652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:36.216382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1048/2128] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:36 2022] DELETE /identity/v3/projects/fab43244f2a043f988668489b6b7f6c4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:36.222213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e43f-c788-4a5e-a912-324c7baee561 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.222722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e43f-c788-4a5e-a912-324c7baee561 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.222815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e43f-c788-4a5e-a912-324c7baee561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.260305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6795e43f-c788-4a5e-a912-324c7baee561 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:36.261986 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6795e43f-c788-4a5e-a912-324c7baee561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:36.352679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1081/2129] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:36.355206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c504e28-57e8-403e-b8eb-99cb35399939 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.355674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c504e28-57e8-403e-b8eb-99cb35399939 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.356099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c504e28-57e8-403e-b8eb-99cb35399939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.386404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8c504e28-57e8-403e-b8eb-99cb35399939 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:36.388529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8c504e28-57e8-403e-b8eb-99cb35399939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:36.412302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1049/2130] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:36.417231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795e43f-c788-4a5e-a912-324c7baee561 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:36.433161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795e43f-c788-4a5e-a912-324c7baee561 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:36.439649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:36.440581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.441036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.441680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.442606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:36.443116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:36.447434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:36.475632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1082/2131] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:36.476877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c504e28-57e8-403e-b8eb-99cb35399939 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:36.492268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c504e28-57e8-403e-b8eb-99cb35399939 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:36.499108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:36.500284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.500791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.501497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.502988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:36.505414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:36.531385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:36.560610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1050/2132] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:36 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:36.561613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:36.574592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72259347-318a-4a88-8cda-e9d20255e42e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:36.624800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:36.673369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:36.673988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.675098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.675098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.675352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:36.678857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:36.761601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1083/2133] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:36 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:36.766830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:36.783828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c6d6-7643-41ea-8a3e-c5203046ea68 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:36.803201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:36.803827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.804958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.808436 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.811013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:36.818872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:36.850243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1051/2134] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:36.851921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:36.866610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318f74b-32e7-480f-8ce3-e04d6462cad2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:36.881350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8284723-8bcc-4027-a5d5-86ba8b658341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:36.884096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8284723-8bcc-4027-a5d5-86ba8b658341 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.884737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8284723-8bcc-4027-a5d5-86ba8b658341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.885620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8284723-8bcc-4027-a5d5-86ba8b658341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:36.887351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8284723-8bcc-4027-a5d5-86ba8b658341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:36.893121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8284723-8bcc-4027-a5d5-86ba8b658341 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:36.924915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:36.925133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1084/2135] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:36.939367 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f61f3-a4be-4db8-bd03-baec217ae870 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:36.954113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:36.954375 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:36.954622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:36.954834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] PATH_INFO: `/v3/projects/4bd40afe56004fec9a625c4de35fd7dc/users/886599283d2f4d71821e39a171687053/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:37.012270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd40afe56004fec9a625c4de35fd7dc, user_id=886599283d2f4d71821e39a171687053, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:37.020165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:37.066274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1052/2136] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:36 2022] PUT /identity/v3/projects/4bd40afe56004fec9a625c4de35fd7dc/users/886599283d2f4d71821e39a171687053/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:37.070799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdd7a9-b32c-4e6e-979f-aea6f14d922e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:37.072467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdd7a9-b32c-4e6e-979f-aea6f14d922e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:37.073923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdd7a9-b32c-4e6e-979f-aea6f14d922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:37.147292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a8bdd7a9-b32c-4e6e-979f-aea6f14d922e None None] MFA Rules not processed for user `886599283d2f4d71821e39a171687053`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:37.149868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a8bdd7a9-b32c-4e6e-979f-aea6f14d922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:37.247379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1085/2137] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:37.274976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:37.277794 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:37.503473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd2d0d-1f0a-446e-bd3c-8c21a4b7878f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:37.512429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:37.513276 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:37.513656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:37.514305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:37.530270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:37.532104 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:37.546851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1053/2138] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:40.883848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193018-d960-4f92-9ea9-751b4a031746 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:40.884361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193018-d960-4f92-9ea9-751b4a031746 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:40.884592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193018-d960-4f92-9ea9-751b4a031746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:40.913876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8a193018-d960-4f92-9ea9-751b4a031746 None None] MFA Rules not processed for user `0eccc82843084cb392b1f15ad2ac9268`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:40.915105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8a193018-d960-4f92-9ea9-751b4a031746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:41.005105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1086/2139] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:41.011335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:41.017452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:41.168857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce269-9afd-4c27-a2fa-6e54df7c4f48 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:41.173514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:41.174133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:41.174415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:41.174689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:41.175329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:41.177960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:41.209474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1054/2140] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:41 2022] POST /identity/v3/projects => generated 373 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:41.213952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193018-d960-4f92-9ea9-751b4a031746 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:41.230265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193018-d960-4f92-9ea9-751b4a031746 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:41.288030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:41.306296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:41.307060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:41.307404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:41.307726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:41.308557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:41.310583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:41.337245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1087/2141] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:41 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:41.342474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:41.352956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b64911-6e72-4d84-822e-b3953967eae5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:41.358577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:41.359248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:41.359533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:41.359806 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:41.360458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:41.362124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:41.373570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1055/2142] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:41.377558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:41.389269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec87706-0b52-4dbf-af44-d4bcd3b89741 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:41.394988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:41.395773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:41.396126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:41.396397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] PATH_INFO: `/v3/projects/6ea109160a154ce6a71660bdcf306133/users/82bda11d02fd4147a24800327159c539/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:41.417406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea109160a154ce6a71660bdcf306133, user_id=82bda11d02fd4147a24800327159c539, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:41.420113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:41.435418 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1088/2143] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:41 2022] PUT /identity/v3/projects/6ea109160a154ce6a71660bdcf306133/users/82bda11d02fd4147a24800327159c539/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:41.439505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:41.450900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c8d87-562c-49cf-b307-c22950e47afb admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:41.487412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:41.519091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:41.528635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:41.528996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:41.529363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:41.530163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:41.532046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:41.545858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1056/2144] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:41.553965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:41.566704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c84067-a58e-48f5-912f-a8162470ae44 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:41.577877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28c45c-7628-4cc9-a371-1995b8d1eac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:41.578683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28c45c-7628-4cc9-a371-1995b8d1eac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:41.579065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28c45c-7628-4cc9-a371-1995b8d1eac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:41.579458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28c45c-7628-4cc9-a371-1995b8d1eac7 admin admin] PATH_INFO: `/v3/projects/6ea109160a154ce6a71660bdcf306133/users/82bda11d02fd4147a24800327159c539/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:41.581632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28c45c-7628-4cc9-a371-1995b8d1eac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea109160a154ce6a71660bdcf306133, user_id=82bda11d02fd4147a24800327159c539, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:41.584455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28c45c-7628-4cc9-a371-1995b8d1eac7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:41.597035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1089/2145] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:41 2022] PUT /identity/v3/projects/6ea109160a154ce6a71660bdcf306133/users/82bda11d02fd4147a24800327159c539/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:42.282873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:42.307785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c74d-161c-446a-a36c-9caad9f9524a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:42.462002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:42.462664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:42.462946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:42.463753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] PATH_INFO: `/v3/users/886599283d2f4d71821e39a171687053` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:42.465008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=886599283d2f4d71821e39a171687053)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:42.466964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:42.582793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886599283d2f4d71821e39a171687053'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:42.611157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886599283d2f4d71821e39a171687053'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:42.627626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886599283d2f4d71821e39a171687053'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:42.641424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886599283d2f4d71821e39a171687053'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:42.642257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886599283d2f4d71821e39a171687053'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:42.661530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1057/2146] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:42 2022] DELETE /identity/v3/users/886599283d2f4d71821e39a171687053 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:42.664436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:42.665068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:42.665581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:42.724357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:42.727097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:42.895914 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1090/2147] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:42.911695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:42.913188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:42.991639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08452a0-fd2e-42ef-b3a2-767366852ad0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:42.995498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df840dba-b0f3-4bdd-808e-8e75bb766f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:42.996239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840dba-b0f3-4bdd-808e-8e75bb766f9a service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:42.996611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840dba-b0f3-4bdd-808e-8e75bb766f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:42.996976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840dba-b0f3-4bdd-808e-8e75bb766f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:43.006044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:43.017246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df840dba-b0f3-4bdd-808e-8e75bb766f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:43.017246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df840dba-b0f3-4bdd-808e-8e75bb766f9a service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:43.058283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1058/2148] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:43.594210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:43.597550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:43.621067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1749cd-80d1-4416-a0e4-908fe9055fb9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:43.628296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:43.629841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:43.630379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:43.632318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] PATH_INFO: `/v3/projects/4bd40afe56004fec9a625c4de35fd7dc` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:43.634882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd40afe56004fec9a625c4de35fd7dc)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:43.636724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:43.711786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd40afe56004fec9a625c4de35fd7dc', 'request_id': 'req-311e4d85-db30-4ff9-9f66-d1fa3f74e925'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:43.720552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd40afe56004fec9a625c4de35fd7dc', 'request_id': 'req-311e4d85-db30-4ff9-9f66-d1fa3f74e925'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:43.750623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd40afe56004fec9a625c4de35fd7dc', 'request_id': 'req-311e4d85-db30-4ff9-9f66-d1fa3f74e925'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:43.778626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] The token cache is being invalidate because project 4bd40afe56004fec9a625c4de35fd7dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:43.779276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:43.780365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-311e4d85-db30-4ff9-9f66-d1fa3f74e925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:43.782435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1091/2149] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:43 2022] DELETE /identity/v3/projects/4bd40afe56004fec9a625c4de35fd7dc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:43.787232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fa25-96fb-4a86-8398-e58e1adfe491 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:43.788558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fa25-96fb-4a86-8398-e58e1adfe491 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:43.789107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fa25-96fb-4a86-8398-e58e1adfe491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:43.834231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-79e3fa25-96fb-4a86-8398-e58e1adfe491 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:43.835731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-79e3fa25-96fb-4a86-8398-e58e1adfe491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:43.993069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1059/2150] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:58:44.003559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c282d-8afa-4806-9703-89def4ae5893 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.004211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c282d-8afa-4806-9703-89def4ae5893 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.004645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c282d-8afa-4806-9703-89def4ae5893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.060580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-df1c282d-8afa-4806-9703-89def4ae5893 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:44.061890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-df1c282d-8afa-4806-9703-89def4ae5893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:44.088641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1092/2151] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:44.093019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3fa25-96fb-4a86-8398-e58e1adfe491 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:44.106761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3fa25-96fb-4a86-8398-e58e1adfe491 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:44.118924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:44.119995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.120279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.124766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.126963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:44.127461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:44.129237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:44.172494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1060/2152] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:44.173533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c282d-8afa-4806-9703-89def4ae5893 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:44.200833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c282d-8afa-4806-9703-89def4ae5893 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:44.239734 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:44.240370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.240775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.241258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.242064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:44.245439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:44.266629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:44.293759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1093/2153] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:44 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:44.298606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:44.319350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d39127-871e-4cc3-b865-44ed9c8a70fe admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:44.465806 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:44.466518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.466897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.467506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.468029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:44.471396 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:44.529249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1061/2154] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:44 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:44.531494 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:44.547233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae090c13-3068-41ae-8318-5755f4a45dae admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:44.555241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:44.556354 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.557974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.558720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.559794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:44.562796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:44.578866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1094/2155] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:44.583074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:44.611386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fb2ed-e418-4eaf-a49b-08a282ca243e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:44.631875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddcf04-0261-40d1-9b5a-b5bb6d16798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:44.632674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddcf04-0261-40d1-9b5a-b5bb6d16798e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.632962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddcf04-0261-40d1-9b5a-b5bb6d16798e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.633256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddcf04-0261-40d1-9b5a-b5bb6d16798e admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.633917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddcf04-0261-40d1-9b5a-b5bb6d16798e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:44.636975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddcf04-0261-40d1-9b5a-b5bb6d16798e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:44.684620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1062/2156] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:58:44.689244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:44.703806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a540801-c615-40d8-8c19-af1eb0232320 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:44.709946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:44.711154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.711726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.712214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] PATH_INFO: `/v3/projects/be1d452583bf4052b9da902950603e82/users/2c7d62693fd24d4cbe678bd995bb6b04/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.733315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1d452583bf4052b9da902950603e82, user_id=2c7d62693fd24d4cbe678bd995bb6b04, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:44.737419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:44.768649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1095/2157] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:44 2022] PUT /identity/v3/projects/be1d452583bf4052b9da902950603e82/users/2c7d62693fd24d4cbe678bd995bb6b04/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:44.772561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778eea-b0c3-436d-85db-3623e0e35af9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:44.772851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778eea-b0c3-436d-85db-3623e0e35af9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:44.773124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778eea-b0c3-436d-85db-3623e0e35af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:44.843781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-63778eea-b0c3-436d-85db-3623e0e35af9 None None] MFA Rules not processed for user `2c7d62693fd24d4cbe678bd995bb6b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:44.845688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-63778eea-b0c3-436d-85db-3623e0e35af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:44.987751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:45.030256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1063/2158] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:45.031506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:45.159036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a8ac6-4f41-4e56-98d9-ebb7795c5671 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:45.173084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:45.173729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:45.174009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:45.174650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:45.174913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:45.176887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:45.250890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1096/2159] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:45 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:45.254981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63778eea-b0c3-436d-85db-3623e0e35af9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:45.272222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63778eea-b0c3-436d-85db-3623e0e35af9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:45.402280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:45.402919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:45.403223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:45.403561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:45.405292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:45.407159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:45.473045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1064/2160] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:45 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:45.477374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:45.496501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d52b8-b033-405b-bc05-facd67bef1c1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:45.521834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:45.522503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:45.522830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:45.523192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:45.523942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:45.526122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:45.557646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1097/2161] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:45.560985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:45.578312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d9192-2222-4287-b651-645f9c826c20 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:45.585184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d102aba-3760-4580-aed3-51381bf46807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:45.586029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d102aba-3760-4580-aed3-51381bf46807 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:45.586474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d102aba-3760-4580-aed3-51381bf46807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:45.586897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d102aba-3760-4580-aed3-51381bf46807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:45.587829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d102aba-3760-4580-aed3-51381bf46807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:45.590422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d102aba-3760-4580-aed3-51381bf46807 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:45.617221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1065/2162] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:45.617448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:45.627404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71609-e699-4835-98f7-93c2f49dc582 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:45.642363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:45.643329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:45.646105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:45.646572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] PATH_INFO: `/v3/projects/f3cd4decc9d14809abb992456001807d/users/e27ecf122e4c4c1295abcec563cd5921/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:45.682450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cd4decc9d14809abb992456001807d, user_id=e27ecf122e4c4c1295abcec563cd5921, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:45.685367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:45.714365 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1098/2163] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:45 2022] PUT /identity/v3/projects/f3cd4decc9d14809abb992456001807d/users/e27ecf122e4c4c1295abcec563cd5921/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:45.716956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a99061-11bc-4977-a18e-cac33ac15f8b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:45.717260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a99061-11bc-4977-a18e-cac33ac15f8b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:45.717602 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a99061-11bc-4977-a18e-cac33ac15f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:45.808594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-89a99061-11bc-4977-a18e-cac33ac15f8b None None] MFA Rules not processed for user `e27ecf122e4c4c1295abcec563cd5921`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:45.811109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-89a99061-11bc-4977-a18e-cac33ac15f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:45.949737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1066/2164] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:45.954535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:46.097355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fca4a-2912-4d4b-919d-48b258f0de69 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:46.103461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:46.104126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.104406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.104677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.105351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:46.107276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:46.152436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:46.175404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1099/2165] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:45 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:46.179443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a99061-11bc-4977-a18e-cac33ac15f8b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:46.202110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a99061-11bc-4977-a18e-cac33ac15f8b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:46.338933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:46.339633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.339918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.340757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.341071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:46.344796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:46.366134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:46.394769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1067/2166] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:46 2022] POST /identity/v3/users => generated 299 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:46.398636 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:46.414708 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71c27-74dc-4f29-a55f-fee2231c5514 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:46.427248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:46.428162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.428643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.429094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.429952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:46.436724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:46.466283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1100/2167] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:46.471983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:46.500028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059fd92-85e4-4351-b2bd-25ebf23b4ea3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:46.506442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:46.506684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.506879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.507262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] PATH_INFO: `/v3/projects/6fb69bede6bb4f1d88862f072a37bb1a/users/8657c8d8cec04810bc155184e092ae27/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.560453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb69bede6bb4f1d88862f072a37bb1a, user_id=8657c8d8cec04810bc155184e092ae27, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:46.566447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:46.602901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1068/2168] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:46 2022] PUT /identity/v3/projects/6fb69bede6bb4f1d88862f072a37bb1a/users/8657c8d8cec04810bc155184e092ae27/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:46.607429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:46.623968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86d343-47c9-406f-b10d-cdb2ca2c3c86 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:46.754705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:46.756037 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.756632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.757077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.758326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:46.763952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:46.779080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1101/2169] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:46.783877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:46.816698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbf807-0796-44cb-8fc7-07439c8e2110 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:46.830894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88d041-6092-446f-91e6-522a798ec337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:46.832108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88d041-6092-446f-91e6-522a798ec337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.832690 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88d041-6092-446f-91e6-522a798ec337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.833230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88d041-6092-446f-91e6-522a798ec337 admin admin] PATH_INFO: `/v3/projects/6fb69bede6bb4f1d88862f072a37bb1a/users/8657c8d8cec04810bc155184e092ae27/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.837947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88d041-6092-446f-91e6-522a798ec337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb69bede6bb4f1d88862f072a37bb1a, user_id=8657c8d8cec04810bc155184e092ae27, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:46.842347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88d041-6092-446f-91e6-522a798ec337 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:46.854377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:46.858440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1069/2170] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:46 2022] PUT /identity/v3/projects/6fb69bede6bb4f1d88862f072a37bb1a/users/8657c8d8cec04810bc155184e092ae27/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:46.859072 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:46.873593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.874087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.874528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.891142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e3dbd-f321-49e7-9477-88590b9844d0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:46.920594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] MFA Rules not processed for user `8657c8d8cec04810bc155184e092ae27`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:46.922132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:46.987544 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:46.988009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:46.988296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:46.988569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] PATH_INFO: `/v3/users/36f54af102e14a1294833bb37d36348a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:46.990337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36f54af102e14a1294833bb37d36348a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:46.995198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:47.030965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1070/2171] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:47.043345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:47.045980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:47.091993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36f54af102e14a1294833bb37d36348a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:47.101651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36f54af102e14a1294833bb37d36348a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:47.118194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36f54af102e14a1294833bb37d36348a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:47.119621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36f54af102e14a1294833bb37d36348a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:47.145537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7d21d19d-3eb7-4311-93cd-e374c9026b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36f54af102e14a1294833bb37d36348a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:47.148198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:47.155727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1102/2172] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:46 2022] DELETE /identity/v3/users/36f54af102e14a1294833bb37d36348a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:47.163012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:47.163499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:47.163965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:47.203035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:47.206119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:47.279987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d1cc4-814e-49eb-a75d-c41d43f5ef06 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:47.304936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1103/2173] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:47.316065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:47.319620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:47.390547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fea44-45e5-44d7-9ebc-d22e5a25cc93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:47.391639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fea44-45e5-44d7-9ebc-d22e5a25cc93 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:47.392098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fea44-45e5-44d7-9ebc-d22e5a25cc93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:47.392524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fea44-45e5-44d7-9ebc-d22e5a25cc93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:47.395153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fb3fea44-45e5-44d7-9ebc-d22e5a25cc93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:47.413620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:47.421982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73f54b-176b-44c2-bbe3-323bbb10cc5f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:47.428923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:47.429951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:47.430417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:47.430873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:47.444756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:47.448130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:47.480004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1104/2174] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:47.559113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fea44-45e5-44d7-9ebc-d22e5a25cc93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:47.564058 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fea44-45e5-44d7-9ebc-d22e5a25cc93 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:47.605002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1071/2175] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:47.881725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:47.883786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:47.907775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf32d9-3304-4d8f-8ce8-7a219d3b2ee8 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:47.914029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:47.921837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:47.922370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:47.922857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] PATH_INFO: `/v3/projects/090e4bcba84f427d8c8aa7d1d11fef56` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:47.924270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090e4bcba84f427d8c8aa7d1d11fef56)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:47.926320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:48.059391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090e4bcba84f427d8c8aa7d1d11fef56', 'request_id': 'req-d4bdd811-34fb-4747-b173-6bab756caef6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:48.103791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090e4bcba84f427d8c8aa7d1d11fef56', 'request_id': 'req-d4bdd811-34fb-4747-b173-6bab756caef6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:48.124517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090e4bcba84f427d8c8aa7d1d11fef56', 'request_id': 'req-d4bdd811-34fb-4747-b173-6bab756caef6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:48.156057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] The token cache is being invalidate because project 090e4bcba84f427d8c8aa7d1d11fef56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:48.156526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:48.158607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d4bdd811-34fb-4747-b173-6bab756caef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:48.186769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1105/2176] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:47 2022] DELETE /identity/v3/projects/090e4bcba84f427d8c8aa7d1d11fef56 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:48.193423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c4ce-1267-4608-9900-15e061b52a8f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.193927 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c4ce-1267-4608-9900-15e061b52a8f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.194118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c4ce-1267-4608-9900-15e061b52a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.220252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3168c4ce-1267-4608-9900-15e061b52a8f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:48.221722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3168c4ce-1267-4608-9900-15e061b52a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:48.335350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff58d84-a058-43a9-9113-94facf9ebb77 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.336068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff58d84-a058-43a9-9113-94facf9ebb77 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.336724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff58d84-a058-43a9-9113-94facf9ebb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.341623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1072/2177] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:48.360762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5ff58d84-a058-43a9-9113-94facf9ebb77 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:48.362333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5ff58d84-a058-43a9-9113-94facf9ebb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:48.373560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1106/2178] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:48.376766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168c4ce-1267-4608-9900-15e061b52a8f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:48.389765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168c4ce-1267-4608-9900-15e061b52a8f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:48.395741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:48.396537 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.396960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.397777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.398948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:58:48.399420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:48.401647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:48.416450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1073/2179] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:58:48.421079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff58d84-a058-43a9-9113-94facf9ebb77 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:48.432255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff58d84-a058-43a9-9113-94facf9ebb77 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:48.439082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:48.440089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.440541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.440968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.441796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:48.443921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:48.464998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:48.488989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1107/2180] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:58:48 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:58:48.493927 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:48.506938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a9b01-f4db-4914-9821-ecd3dd252cb1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:48.633156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:48.634168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.636290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.636688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.637471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:48.645678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:48.703663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:48.711741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1074/2181] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:58:48 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:58:48.721001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753b9df-9469-4f14-a0d3-e044a74b042f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:48.730826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:48.731650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.732034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.732409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.733351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:48.736065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:48.753420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:48.753954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1108/2182] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:48.767213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c59628-d34b-409b-bc54-ee439a954a18 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:48.774444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac52a3d-4ff9-4cde-a59a-bebcea62c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:48.775087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac52a3d-4ff9-4cde-a59a-bebcea62c0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.775508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac52a3d-4ff9-4cde-a59a-bebcea62c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.775886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac52a3d-4ff9-4cde-a59a-bebcea62c0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.776954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac52a3d-4ff9-4cde-a59a-bebcea62c0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:48.779358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac52a3d-4ff9-4cde-a59a-bebcea62c0f0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:48.792957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1075/2183] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:58:48.797393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:48.809920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ddeec-5e67-445b-9648-1225ece77996 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:48.815263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:48.816245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.816630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.817004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] PATH_INFO: `/v3/projects/6eb66b31431f4873bce3ebac6f816b4f/users/1de712ff32864f7798599e8b0469c115/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.835637 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb66b31431f4873bce3ebac6f816b4f, user_id=1de712ff32864f7798599e8b0469c115, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:48.838845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:48.861559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1109/2184] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:58:48 2022] PUT /identity/v3/projects/6eb66b31431f4873bce3ebac6f816b4f/users/1de712ff32864f7798599e8b0469c115/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:58:48.873399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:48.873767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:48.874117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:48.906814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:48.910624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] MFA Rules not processed for user `1de712ff32864f7798599e8b0469c115`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:48.912462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:49.002044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1076/2185] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:49.031383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:49.033191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:49.122095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4726ac-a950-4b19-ab1f-a88d43b69f01 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:49.126757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c086f-46de-475a-a000-8389b40dff95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:49.127371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c086f-46de-475a-a000-8389b40dff95 service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:49.127647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c086f-46de-475a-a000-8389b40dff95 service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:49.127908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c086f-46de-475a-a000-8389b40dff95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:49.137103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c086f-46de-475a-a000-8389b40dff95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:49.138564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c086f-46de-475a-a000-8389b40dff95 service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:49.149255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1110/2186] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:49.493413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:49.494877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:49.587598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde59b6-a2c8-43c3-80b9-4e159a7317be None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:49.591968 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:49.592587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:49.592919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:49.593393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:49.596235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:49.718719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:49.721830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:49.739409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1077/2187] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:50.392032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:50.392524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:50.392615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:50.481909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] MFA Rules not processed for user `82bda11d02fd4147a24800327159c539`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:50.483712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:50.691924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1111/2188] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:50.707911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:50.709921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:50.842902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:50.865947 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b679e4-39c9-4993-a408-a63cb62d7297 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:50.873446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72589b55-0daa-4fe1-9efa-d886beea304c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:50.874097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589b55-0daa-4fe1-9efa-d886beea304c service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:50.874380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589b55-0daa-4fe1-9efa-d886beea304c service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:50.874652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589b55-0daa-4fe1-9efa-d886beea304c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:50.876753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-72589b55-0daa-4fe1-9efa-d886beea304c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:50.964797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72589b55-0daa-4fe1-9efa-d886beea304c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:50.966866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72589b55-0daa-4fe1-9efa-d886beea304c service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:50.984649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1078/2189] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:50.985756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:50.987372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:51.005004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2403b45-5697-4dfb-ab6a-ad14f08dbad4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:51.009234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:51.010370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:51.010654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:51.010925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:51.013024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:51.072038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:51.098897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:51.100367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:51.111671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1112/2190] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:58:52.539896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2f43a-6597-40b5-a27b-2d89d56abfa5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:52.540383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2f43a-6597-40b5-a27b-2d89d56abfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:52.540497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2f43a-6597-40b5-a27b-2d89d56abfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:52.569950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b7b2f43a-6597-40b5-a27b-2d89d56abfa5 None None] MFA Rules not processed for user `0eccc82843084cb392b1f15ad2ac9268`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:52.571654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b7b2f43a-6597-40b5-a27b-2d89d56abfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:52.581579 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1079/2191] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:57.300185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:57.301728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:57.377135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6849d0f-e3fa-4cc7-8bcd-6ee61084a756 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:57.382681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:57.383792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:57.384175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:57.384461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] PATH_INFO: `/v3/users/2c7d62693fd24d4cbe678bd995bb6b04` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:57.385713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7d62693fd24d4cbe678bd995bb6b04)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:57.387731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:57.454064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7d62693fd24d4cbe678bd995bb6b04'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:57.460978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7d62693fd24d4cbe678bd995bb6b04'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:57.473604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7d62693fd24d4cbe678bd995bb6b04'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:57.474288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7d62693fd24d4cbe678bd995bb6b04'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:57.483140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7d62693fd24d4cbe678bd995bb6b04'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:57.489359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1113/2192] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:57 2022] DELETE /identity/v3/users/2c7d62693fd24d4cbe678bd995bb6b04 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:57.493843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:57.494349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:57.494442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:57.514487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:58:57.515700 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:57.577137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1080/2193] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:58:57.586938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:57.588882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:57.667854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2bff2-bd9e-43b2-aa78-2b714896ac47 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:57.671660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:57.672251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:57.672531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:57.672807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:57.682710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:57.684145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:57.693579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1114/2194] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:58:57.979738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:57.981529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:58.002070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930cb0dd-48b6-47c4-b770-cbb5f204e441 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:58.007344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:58.007988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:58.008269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:58.008540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] PATH_INFO: `/v3/projects/be1d452583bf4052b9da902950603e82` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:58.013014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1d452583bf4052b9da902950603e82)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:58.020797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:58.113561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1d452583bf4052b9da902950603e82', 'request_id': 'req-582310f9-e77e-48ad-b152-468a98e5c42b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.121242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1d452583bf4052b9da902950603e82', 'request_id': 'req-582310f9-e77e-48ad-b152-468a98e5c42b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.127737 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1d452583bf4052b9da902950603e82', 'request_id': 'req-582310f9-e77e-48ad-b152-468a98e5c42b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.146234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] The token cache is being invalidate because project be1d452583bf4052b9da902950603e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:58.146800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.147588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.149481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1081/2195] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:57 2022] DELETE /identity/v3/projects/be1d452583bf4052b9da902950603e82 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:58.153717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:58.162367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:58.174821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423041a-c521-44fb-811c-8796a5ff1ef5 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:58.282649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:58.283552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:58.283836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:58.284113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] PATH_INFO: `/v3/users/e27ecf122e4c4c1295abcec563cd5921` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:58.285324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27ecf122e4c4c1295abcec563cd5921)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:58.287295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:58.351051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27ecf122e4c4c1295abcec563cd5921'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.360482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27ecf122e4c4c1295abcec563cd5921'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.375675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27ecf122e4c4c1295abcec563cd5921'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.376002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27ecf122e4c4c1295abcec563cd5921'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.386882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27ecf122e4c4c1295abcec563cd5921'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.400444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1115/2196] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:58 2022] DELETE /identity/v3/users/e27ecf122e4c4c1295abcec563cd5921 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:58.754284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:58.755899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:58.769634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582310f9-e77e-48ad-b152-468a98e5c42b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:58.778128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:58.834231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:58.835052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:58.835355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:58.835612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] PATH_INFO: `/v3/projects/f3cd4decc9d14809abb992456001807d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:58.836752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cd4decc9d14809abb992456001807d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:58.838766 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:58.889127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cd4decc9d14809abb992456001807d', 'request_id': 'req-505573eb-6ff7-43be-a4ec-7b58fd21b328'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.897656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cd4decc9d14809abb992456001807d', 'request_id': 'req-505573eb-6ff7-43be-a4ec-7b58fd21b328'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.904457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cd4decc9d14809abb992456001807d', 'request_id': 'req-505573eb-6ff7-43be-a4ec-7b58fd21b328'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.917259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] The token cache is being invalidate because project f3cd4decc9d14809abb992456001807d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:58.917703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.918338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:58.919972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1082/2197] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:58 2022] DELETE /identity/v3/projects/f3cd4decc9d14809abb992456001807d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:58.923888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:58.925331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:58.938329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0315c-1bc5-4597-8abe-6936a9eabf73 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:59.004635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:59.005308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:59.005597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:59.005869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] PATH_INFO: `/v3/users/8657c8d8cec04810bc155184e092ae27` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:59.007041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8657c8d8cec04810bc155184e092ae27)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:59.009006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:59.051808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:59.073800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8657c8d8cec04810bc155184e092ae27'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.081053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8657c8d8cec04810bc155184e092ae27'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.095290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8657c8d8cec04810bc155184e092ae27'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.096007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8657c8d8cec04810bc155184e092ae27'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.105645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2995b146-0750-49dd-a12a-be97ac01acd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8657c8d8cec04810bc155184e092ae27'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.112846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1116/2198] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:58:58 2022] DELETE /identity/v3/users/8657c8d8cec04810bc155184e092ae27 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:58:59.396312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:58:59.397823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:58:59.411389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505573eb-6ff7-43be-a4ec-7b58fd21b328 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:58:59.476719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:58:59.476719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:58:59.476719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:58:59.476719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] PATH_INFO: `/v3/projects/6fb69bede6bb4f1d88862f072a37bb1a` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:58:59.477387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb69bede6bb4f1d88862f072a37bb1a)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:58:59.478360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:58:59.535921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:58:59.540059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb69bede6bb4f1d88862f072a37bb1a', 'request_id': 'req-82326bd5-f9c3-433a-99bf-c6298045aea3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.548741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb69bede6bb4f1d88862f072a37bb1a', 'request_id': 'req-82326bd5-f9c3-433a-99bf-c6298045aea3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.559219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb69bede6bb4f1d88862f072a37bb1a', 'request_id': 'req-82326bd5-f9c3-433a-99bf-c6298045aea3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.573673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] The token cache is being invalidate because project 6fb69bede6bb4f1d88862f072a37bb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:58:59.574471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.575103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:58:59.577566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1083/2199] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:58:59 2022] DELETE /identity/v3/projects/6fb69bede6bb4f1d88862f072a37bb1a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:09.108547 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=6,keystone:SELECT=19,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:09.342822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2635b19-e5ec-475e-8308-c8e7b61edd5a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:09.343199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2635b19-e5ec-475e-8308-c8e7b61edd5a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:09.343536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2635b19-e5ec-475e-8308-c8e7b61edd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:09.365610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d2635b19-e5ec-475e-8308-c8e7b61edd5a None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:09.367096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d2635b19-e5ec-475e-8308-c8e7b61edd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:09.427516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1117/2200] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:09.471740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:09.473348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:09.561779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82326bd5-f9c3-433a-99bf-c6298045aea3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:09.568234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dd3dc-1542-4999-8739-ded2f28ce966 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:09.568775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dd3dc-1542-4999-8739-ded2f28ce966 service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:09.569056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dd3dc-1542-4999-8739-ded2f28ce966 service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:09.569358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dd3dc-1542-4999-8739-ded2f28ce966 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:09.578065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dd3dc-1542-4999-8739-ded2f28ce966 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:09.579584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dd3dc-1542-4999-8739-ded2f28ce966 service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:09.588222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1084/2201] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 14:59:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 14:59:10.781786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03cef7-03e3-4c4f-b94d-1663db345b5b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:10.782260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03cef7-03e3-4c4f-b94d-1663db345b5b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:10.782542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03cef7-03e3-4c4f-b94d-1663db345b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:10.817690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ac03cef7-03e3-4c4f-b94d-1663db345b5b None None] MFA Rules not processed for user `770105b06e084179a819a54d28b70cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:10.819101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ac03cef7-03e3-4c4f-b94d-1663db345b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:10.890884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1118/2202] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:19.584499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=40,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:20.882815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=50 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:30.818469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:30.818984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:30.819105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:30.842448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:30.843748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:30.850914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1085/2203] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 14:59:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:30.864886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03cef7-03e3-4c4f-b94d-1663db345b5b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:30.876570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03cef7-03e3-4c4f-b94d-1663db345b5b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:30.881618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:30.882378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:30.882824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:30.883096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:30.891590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:30.893228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:30.901173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1119/2204] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 14:59:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 14:59:34.230067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:34.232738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:34.300556 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55eb25-c97e-486b-9ade-67fdb2f1a4bd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:34.305124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:34.305783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:34.306063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:34.306335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] PATH_INFO: `/v3/users/ca360b149ac8402282733edfc4c67f9e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:34.307470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca360b149ac8402282733edfc4c67f9e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:34.309533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:34.356632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca360b149ac8402282733edfc4c67f9e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.368421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca360b149ac8402282733edfc4c67f9e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.375623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca360b149ac8402282733edfc4c67f9e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.383096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca360b149ac8402282733edfc4c67f9e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.383711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d519769a-5825-48be-83c3-048a94196768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca360b149ac8402282733edfc4c67f9e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.389965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1086/2205] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:59:34 2022] DELETE /identity/v3/users/ca360b149ac8402282733edfc4c67f9e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:34.581870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:34.612809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c07dc-1bb0-40b4-b026-b2f0fc9fae98 service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:34.680764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:34.681751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:34.682054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:34.682334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] PATH_INFO: `/v3/projects/bd3cfdafe89243748fe29bef991b2215` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:34.683519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd3cfdafe89243748fe29bef991b2215)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:34.685599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:34.735351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd3cfdafe89243748fe29bef991b2215', 'request_id': 'req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.741746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd3cfdafe89243748fe29bef991b2215', 'request_id': 'req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.747508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd3cfdafe89243748fe29bef991b2215', 'request_id': 'req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.759843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] The token cache is being invalidate because project bd3cfdafe89243748fe29bef991b2215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:59:34.760130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.760749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e85bf1ea-b6f8-461f-be6d-2da9db352fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:34.762850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1120/2206] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:59:34 2022] DELETE /identity/v3/projects/bd3cfdafe89243748fe29bef991b2215 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:34.768593 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae84c0d-081b-4d89-b91c-284e2d654cb6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:34.768883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae84c0d-081b-4d89-b91c-284e2d654cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:34.769160 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae84c0d-081b-4d89-b91c-284e2d654cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:34.789570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bae84c0d-081b-4d89-b91c-284e2d654cb6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:34.790781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bae84c0d-081b-4d89-b91c-284e2d654cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:34.851770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1087/2207] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:34.859463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ce04-91d4-4f20-be38-98b2e170a14f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:34.859857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ce04-91d4-4f20-be38-98b2e170a14f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:34.860136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ce04-91d4-4f20-be38-98b2e170a14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:34.881343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0d17ce04-91d4-4f20-be38-98b2e170a14f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:34.882551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0d17ce04-91d4-4f20-be38-98b2e170a14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:34.891373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1121/2208] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:34.895590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae84c0d-081b-4d89-b91c-284e2d654cb6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:34.906013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae84c0d-081b-4d89-b91c-284e2d654cb6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:34.910785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:34.911385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:34.911663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:34.911934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:34.912577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:59:34.912877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:34.914501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:34.924026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:34.931215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1088/2209] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 14:59:34.932033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17ce04-91d4-4f20-be38-98b2e170a14f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:34.942540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17ce04-91d4-4f20-be38-98b2e170a14f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:34.947588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:34.948194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:34.948474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:34.948742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:34.949433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:34.951895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:34.992345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1122/2210] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:59:34 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:59:34.997731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:35.010967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68894d1-e666-4c6b-9379-f58f2556f883 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:35.080279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:35.080985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:35.081370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:35.081732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:35.082403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:35.084316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:35.113234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1089/2211] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:59:34 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:59:35.114513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:35.126780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8568-ebf8-41b6-a273-f237e737a7b6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:35.132366 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:35.133109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:35.133441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:35.134145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:35.134350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:35.135951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:35.148844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1123/2212] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:59:35.153566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:35.164504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774711d0-7c68-43c9-a6cd-affd81786153 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:35.170281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79947119-1eca-481c-b61f-8900409286b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:35.171023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79947119-1eca-481c-b61f-8900409286b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:35.171377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79947119-1eca-481c-b61f-8900409286b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:35.171745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79947119-1eca-481c-b61f-8900409286b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:35.172481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79947119-1eca-481c-b61f-8900409286b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:35.174214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79947119-1eca-481c-b61f-8900409286b8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:35.188917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1090/2213] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:59:35.192874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:35.203902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713403-ad1a-4044-9d61-99b069a740fe admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:35.208491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:35.209134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:35.209465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:35.209732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] PATH_INFO: `/v3/projects/8214dfad661a496593dba8df6fdc5e4e/users/4bd96d9ff5e74195abb61300565287dc/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:35.225315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8214dfad661a496593dba8df6fdc5e4e, user_id=4bd96d9ff5e74195abb61300565287dc, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:35.228168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:35.241318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1124/2214] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:59:35 2022] PUT /identity/v3/projects/8214dfad661a496593dba8df6fdc5e4e/users/4bd96d9ff5e74195abb61300565287dc/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:35.253542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cae63d-b0c2-4a21-b3a6-7f6cc01630f8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:35.254006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cae63d-b0c2-4a21-b3a6-7f6cc01630f8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:35.254295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cae63d-b0c2-4a21-b3a6-7f6cc01630f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:35.291483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-25cae63d-b0c2-4a21-b3a6-7f6cc01630f8 None None] MFA Rules not processed for user `4bd96d9ff5e74195abb61300565287dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:35.293481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-25cae63d-b0c2-4a21-b3a6-7f6cc01630f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:35.374387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1091/2215] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:35.389243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:35.391671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:35.409729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:35.477851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09973283-3b0e-42b3-b0e7-c3e615f69c15 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:35.485484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:35.486855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:35.487356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:35.487722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:35.499678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:35.501142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:35.513075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1125/2216] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:59:36.155431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:36.155793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:36.156136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:36.176500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:36.177908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:36.249812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1092/2217] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:36.267645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:36.269403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:36.397371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ff7c6-1327-481b-9d1b-988ba1128e3f service glance] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:36.405290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c3ca5-17c2-4441-8f4c-79202f17f5f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:36.405290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c3ca5-17c2-4441-8f4c-79202f17f5f1 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:36.405290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c3ca5-17c2-4441-8f4c-79202f17f5f1 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:36.405290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c3ca5-17c2-4441-8f4c-79202f17f5f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:36.413446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c3ca5-17c2-4441-8f4c-79202f17f5f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:36.415132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c3ca5-17c2-4441-8f4c-79202f17f5f1 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:36.424918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1126/2218] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 14:59:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 14:59:38.575769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:38.577288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:38.594308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:38.675496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09104c24-1e4c-4aa1-964a-92ff2387d995 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:38.683722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:38.684657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:38.685009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:38.685494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] PATH_INFO: `/v3/users/1de712ff32864f7798599e8b0469c115` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:38.686677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de712ff32864f7798599e8b0469c115)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:38.688691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:38.763531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de712ff32864f7798599e8b0469c115'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:38.782740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de712ff32864f7798599e8b0469c115'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:38.790419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de712ff32864f7798599e8b0469c115'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:38.807851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de712ff32864f7798599e8b0469c115'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:38.809184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de712ff32864f7798599e8b0469c115'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:38.825126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1093/2219] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:59:38 2022] DELETE /identity/v3/users/1de712ff32864f7798599e8b0469c115 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:38.826923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:38.827388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:38.827479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:38.860146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:38.861561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:38.940125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1127/2220] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:59:38.955032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:38.956516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:39.094973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eef2ac-6109-4bfe-984b-9e61d3dd0d75 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:39.111225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f742c-f264-4b6c-987e-72b0747f1879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:39.111857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f742c-f264-4b6c-987e-72b0747f1879 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:39.112133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f742c-f264-4b6c-987e-72b0747f1879 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:39.112402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f742c-f264-4b6c-987e-72b0747f1879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:39.128060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f742c-f264-4b6c-987e-72b0747f1879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:39.129692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f742c-f264-4b6c-987e-72b0747f1879 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:39.141601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1094/2221] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:59:39.546279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:39.552803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:39.574860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470f525-94b1-4090-bd22-f080919ae53f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:39.581435 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:39.582012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:39.582290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:39.582561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] PATH_INFO: `/v3/projects/6eb66b31431f4873bce3ebac6f816b4f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:39.583701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb66b31431f4873bce3ebac6f816b4f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:39.585762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:39.638069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:39.646305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb66b31431f4873bce3ebac6f816b4f', 'request_id': 'req-b98b6a49-5ac9-4071-9aa6-421f465241b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:39.655260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb66b31431f4873bce3ebac6f816b4f', 'request_id': 'req-b98b6a49-5ac9-4071-9aa6-421f465241b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:39.661716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb66b31431f4873bce3ebac6f816b4f', 'request_id': 'req-b98b6a49-5ac9-4071-9aa6-421f465241b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:39.676451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] The token cache is being invalidate because project 6eb66b31431f4873bce3ebac6f816b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:59:39.676877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:39.677662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b98b6a49-5ac9-4071-9aa6-421f465241b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:39.685774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1128/2222] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:59:39 2022] DELETE /identity/v3/projects/6eb66b31431f4873bce3ebac6f816b4f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:39.688982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a3cba-4d3e-4ec4-91fd-cbed12414b2d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:39.689489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a3cba-4d3e-4ec4-91fd-cbed12414b2d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:39.689710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a3cba-4d3e-4ec4-91fd-cbed12414b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:39.715511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-803a3cba-4d3e-4ec4-91fd-cbed12414b2d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:39.716842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-803a3cba-4d3e-4ec4-91fd-cbed12414b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:39.752411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:39.795351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1095/2223] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:39.796723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120397f-73b2-4f68-a22c-f5f323a2c41e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:39.797003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120397f-73b2-4f68-a22c-f5f323a2c41e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:39.797325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120397f-73b2-4f68-a22c-f5f323a2c41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:39.821541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b120397f-73b2-4f68-a22c-f5f323a2c41e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:39.823204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b120397f-73b2-4f68-a22c-f5f323a2c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:39.831662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1129/2224] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:39.835752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a3cba-4d3e-4ec4-91fd-cbed12414b2d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:39.860520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a3cba-4d3e-4ec4-91fd-cbed12414b2d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:39.869053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:39.869827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:39.870307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:39.870825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:39.872344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 14:59:39.873423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:39.875313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:39.889234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1096/2225] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 14:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 14:59:39.893514 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120397f-73b2-4f68-a22c-f5f323a2c41e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:39.903803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120397f-73b2-4f68-a22c-f5f323a2c41e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:39.912421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:39.913034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:39.913475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:39.913753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:39.914387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:39.916380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:39.954594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1130/2226] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 14:59:39 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 14:59:39.961038 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:39.977180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41d82-90a3-4b9e-b907-17f20d8b9d59 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:40.053099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:40.053881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.054254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.055881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.055881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:40.057543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:40.089936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1097/2227] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 14:59:39 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 14:59:40.090717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:40.103209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9e54-58a6-4091-b653-af3bdce5f563 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:40.108786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:40.109569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.110061 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.110431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.113131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:40.113131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:40.130216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:40.138305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1131/2228] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 14:59:40.142346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60573ae3-be0b-49ca-921b-771ee8769d86 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:40.154938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc1d48-c0b0-42c8-ab2c-761cb130a402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:40.155689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1d48-c0b0-42c8-ab2c-761cb130a402 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.156060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1d48-c0b0-42c8-ab2c-761cb130a402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.156456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1d48-c0b0-42c8-ab2c-761cb130a402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.157225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc1d48-c0b0-42c8-ab2c-761cb130a402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:40.160574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc1d48-c0b0-42c8-ab2c-761cb130a402 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:40.182906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:40.191982 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1098/2229] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 14:59:40.198164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c2008-9b00-4b8e-81e3-735f1165b5d1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:40.204528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:40.205377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.205754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.206123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] PATH_INFO: `/v3/projects/66c5237102cd455ab218c98b3515ffa1/users/caa83a7a38a045b3b70422f74feb204f/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.225827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c5237102cd455ab218c98b3515ffa1, user_id=caa83a7a38a045b3b70422f74feb204f, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:40.228660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:40.246297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1132/2230] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 14:59:40 2022] PUT /identity/v3/projects/66c5237102cd455ab218c98b3515ffa1/users/caa83a7a38a045b3b70422f74feb204f/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:40.257586 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9768f-8dc9-45c7-9913-d683f42ddd92 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.257979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9768f-8dc9-45c7-9913-d683f42ddd92 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.258257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9768f-8dc9-45c7-9913-d683f42ddd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.346429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6bb9768f-8dc9-45c7-9913-d683f42ddd92 None None] MFA Rules not processed for user `caa83a7a38a045b3b70422f74feb204f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:40.350903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6bb9768f-8dc9-45c7-9913-d683f42ddd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:40.470609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1099/2231] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:40.481834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:40.484831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:40.562733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab731d-156d-44e2-ab86-fdfbaff15899 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:40.569362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137412-3066-4e76-9082-ba94e04a4e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:40.570115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137412-3066-4e76-9082-ba94e04a4e48 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.570508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137412-3066-4e76-9082-ba94e04a4e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.570876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137412-3066-4e76-9082-ba94e04a4e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.581742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137412-3066-4e76-9082-ba94e04a4e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:40.583315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137412-3066-4e76-9082-ba94e04a4e48 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:40.594143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1133/2232] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:59:40.640801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9768f-8dc9-45c7-9913-d683f42ddd92 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:40.707437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:40.765542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9768f-8dc9-45c7-9913-d683f42ddd92 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:40.773862 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:40.774595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.774946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.775293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] PATH_INFO: `/v3/users/caa83a7a38a045b3b70422f74feb204f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.776517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa83a7a38a045b3b70422f74feb204f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:40.779732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:40.842476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa83a7a38a045b3b70422f74feb204f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:40.867668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa83a7a38a045b3b70422f74feb204f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:40.877046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa83a7a38a045b3b70422f74feb204f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:40.890490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa83a7a38a045b3b70422f74feb204f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:40.891547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa83a7a38a045b3b70422f74feb204f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:40.899878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1100/2233] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 14:59:40 2022] DELETE /identity/v3/users/caa83a7a38a045b3b70422f74feb204f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 14:59:40.905683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:40.906260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:40.906260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:40.939450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:40.940671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:41.022871 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1134/2234] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:41.034199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:41.045324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:41.128080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567795b-83cf-463b-ab15-76d45e9dfd10 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:41.133527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa6da1-64d9-48aa-83a3-51e6690ec59c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:41.134154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa6da1-64d9-48aa-83a3-51e6690ec59c service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:41.134439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa6da1-64d9-48aa-83a3-51e6690ec59c service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:41.134712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa6da1-64d9-48aa-83a3-51e6690ec59c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:41.147877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa6da1-64d9-48aa-83a3-51e6690ec59c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:41.149702 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa6da1-64d9-48aa-83a3-51e6690ec59c service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:41.178843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1101/2235] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:59:41.678560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:41.680260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:41.696426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9326b3-d328-4eb3-bb4f-09a93c0bd0f6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:41.704866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:41.705533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:41.706004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:41.706427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] PATH_INFO: `/v3/projects/66c5237102cd455ab218c98b3515ffa1` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:41.708511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c5237102cd455ab218c98b3515ffa1)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:41.710635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:41.714402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:41.783295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c5237102cd455ab218c98b3515ffa1', 'request_id': 'req-216a2aff-057f-4163-96fd-da2811ff091f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:41.789767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c5237102cd455ab218c98b3515ffa1', 'request_id': 'req-216a2aff-057f-4163-96fd-da2811ff091f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:41.798805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c5237102cd455ab218c98b3515ffa1', 'request_id': 'req-216a2aff-057f-4163-96fd-da2811ff091f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:41.813648 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] The token cache is being invalidate because project 66c5237102cd455ab218c98b3515ffa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 14:59:41.813967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:41.814595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 14:59:41.829329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1135/2236] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 14:59:41 2022] DELETE /identity/v3/projects/66c5237102cd455ab218c98b3515ffa1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 14:59:43.950296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c993f81-dcdb-49ca-bc38-7759a176794a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:43.950756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c993f81-dcdb-49ca-bc38-7759a176794a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:43.950854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c993f81-dcdb-49ca-bc38-7759a176794a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:43.978048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5c993f81-dcdb-49ca-bc38-7759a176794a None None] MFA Rules not processed for user `cd6c40dd492e40599324ef05011895f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:43.979688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5c993f81-dcdb-49ca-bc38-7759a176794a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:44.069731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:44.089887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1102/2237] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:59:48.164860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:48.166415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:48.233296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a2aff-057f-4163-96fd-da2811ff091f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:48.237150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0382fc-857a-4089-8314-90114f1a2ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:48.237793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0382fc-857a-4089-8314-90114f1a2ead service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:48.238141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0382fc-857a-4089-8314-90114f1a2ead service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:48.238482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0382fc-857a-4089-8314-90114f1a2ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:48.240268 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-da0382fc-857a-4089-8314-90114f1a2ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:48.320210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0382fc-857a-4089-8314-90114f1a2ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:48.321806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0382fc-857a-4089-8314-90114f1a2ead service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:48.334626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1136/2238] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 14:59:53.052905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3c88f-4136-433f-919c-21889b96fc21 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.053484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3c88f-4136-433f-919c-21889b96fc21 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.053609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3c88f-4136-433f-919c-21889b96fc21 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.055470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1103/2239] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:59:53.063422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88895-fd08-403a-95a9-bf4025545d2a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.064335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88895-fd08-403a-95a9-bf4025545d2a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.064696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a88895-fd08-403a-95a9-bf4025545d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.091951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c0a88895-fd08-403a-95a9-bf4025545d2a None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:53.093449 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c0a88895-fd08-403a-95a9-bf4025545d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:53.145925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 14:59:53.188292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1137/2240] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 14:59:53.201937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3c88f-4136-433f-919c-21889b96fc21 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:53.203634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ebc3c88f-4136-433f-919c-21889b96fc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:53.264881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3c88f-4136-433f-919c-21889b96fc21 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:53.268647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdba99-81ce-4fa3-a50c-1765204e0536 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:53.269280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdba99-81ce-4fa3-a50c-1765204e0536 service glance] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.269559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdba99-81ce-4fa3-a50c-1765204e0536 service glance] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.269828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdba99-81ce-4fa3-a50c-1765204e0536 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.280713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdba99-81ce-4fa3-a50c-1765204e0536 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:53.282195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdba99-81ce-4fa3-a50c-1765204e0536 service glance] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:53.293707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1104/2241] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:59:53.362723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e483ab-3544-4da4-8420-f15ce8ebb39b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.363009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e483ab-3544-4da4-8420-f15ce8ebb39b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.363281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e483ab-3544-4da4-8420-f15ce8ebb39b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.365056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1138/2242] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:59:53.372167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff94d-c505-4bc9-8f7b-5a27c209d1ab None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.372460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff94d-c505-4bc9-8f7b-5a27c209d1ab None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.374907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff94d-c505-4bc9-8f7b-5a27c209d1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.409231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f80ff94d-c505-4bc9-8f7b-5a27c209d1ab None None] MFA Rules not processed for user `f9edd8cf448e42fd8e888c8f531fa70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:53.410458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f80ff94d-c505-4bc9-8f7b-5a27c209d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:53.417300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1105/2243] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:53.430638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e483ab-3544-4da4-8420-f15ce8ebb39b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:53.442970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e483ab-3544-4da4-8420-f15ce8ebb39b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:53.446822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c704d74-0565-48df-b4fb-eb36b0863f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:53.447419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c704d74-0565-48df-b4fb-eb36b0863f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.447694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c704d74-0565-48df-b4fb-eb36b0863f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.447965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c704d74-0565-48df-b4fb-eb36b0863f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.459171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c704d74-0565-48df-b4fb-eb36b0863f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:53.460808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c704d74-0565-48df-b4fb-eb36b0863f92 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:53.471442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1139/2244] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:59:53.563507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13bdef-0a1e-4561-b1b8-45b7fd5f0671 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.563872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13bdef-0a1e-4561-b1b8-45b7fd5f0671 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.564438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13bdef-0a1e-4561-b1b8-45b7fd5f0671 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.566454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1106/2245] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 14:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 14:59:53.572002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b6df-8028-4caa-b523-73d08965982e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.572355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b6df-8028-4caa-b523-73d08965982e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.572695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b6df-8028-4caa-b523-73d08965982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.593988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-80c7b6df-8028-4caa-b523-73d08965982e None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:53.595184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-80c7b6df-8028-4caa-b523-73d08965982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:53.604495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1140/2246] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:53.615669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13bdef-0a1e-4561-b1b8-45b7fd5f0671 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:53.627047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13bdef-0a1e-4561-b1b8-45b7fd5f0671 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:53.630894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:53.631485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:53.631759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:53.632026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:53.642328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:53.643742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:53.654869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1107/2247] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 14:59:57.667823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:57.668801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:57.668801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:57.691137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 14:59:57.692441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:57.752711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1141/2248] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 14:59:57.766595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 14:59:57.768190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 14:59:57.852697 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef976c-30a1-45ce-8c12-8779e7115e5c service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 14:59:57.858421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d28614-fca0-44b3-87f7-5c396bafa36d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 14:59:57.859097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d28614-fca0-44b3-87f7-5c396bafa36d service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 14:59:57.859380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d28614-fca0-44b3-87f7-5c396bafa36d service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 14:59:57.859648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d28614-fca0-44b3-87f7-5c396bafa36d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 14:59:57.867680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d28614-fca0-44b3-87f7-5c396bafa36d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 14:59:57.869314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d28614-fca0-44b3-87f7-5c396bafa36d service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 14:59:57.878299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1108/2249] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 14:59:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:00:07.078924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:07.080374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:07.150867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb623fe1-2e97-4b39-92f2-d01ce8096d6b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:07.155627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:07.156255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:07.156531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:07.156803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] PATH_INFO: `/v3/users/4bd96d9ff5e74195abb61300565287dc` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:07.158009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd96d9ff5e74195abb61300565287dc)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:07.159945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:07.207136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd96d9ff5e74195abb61300565287dc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.213911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd96d9ff5e74195abb61300565287dc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.219147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:07.227982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd96d9ff5e74195abb61300565287dc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.228658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd96d9ff5e74195abb61300565287dc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.235801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd96d9ff5e74195abb61300565287dc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.241906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1142/2250] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:00:07 2022] DELETE /identity/v3/users/4bd96d9ff5e74195abb61300565287dc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:07.247200 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:07.247742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:07.247742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:07.268157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:07.269319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:07.326172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1109/2251] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:07.335811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:07.337235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:07.420265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb35e7d-2de7-43da-ad68-ea1468f7dcdf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:07.425895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41c6d5-6de7-494b-9782-2a49240a164e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:07.426633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41c6d5-6de7-494b-9782-2a49240a164e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:07.427157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41c6d5-6de7-494b-9782-2a49240a164e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:07.427527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41c6d5-6de7-494b-9782-2a49240a164e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:07.440133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41c6d5-6de7-494b-9782-2a49240a164e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:07.441852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41c6d5-6de7-494b-9782-2a49240a164e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:07.465786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1143/2252] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:00:07.762199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:07.763660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:07.779340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88968e-c0b9-4ff8-b171-de97a085784f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:07.784282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:07.784916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:07.785246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:07.785526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] PATH_INFO: `/v3/projects/8214dfad661a496593dba8df6fdc5e4e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:07.786657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8214dfad661a496593dba8df6fdc5e4e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:07.788569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:07.796552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:07.852336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8214dfad661a496593dba8df6fdc5e4e', 'request_id': 'req-a7211b7c-9413-4d3b-8823-8ad00987f1a5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.859973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8214dfad661a496593dba8df6fdc5e4e', 'request_id': 'req-a7211b7c-9413-4d3b-8823-8ad00987f1a5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.865779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8214dfad661a496593dba8df6fdc5e4e', 'request_id': 'req-a7211b7c-9413-4d3b-8823-8ad00987f1a5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.878262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] The token cache is being invalidate because project 8214dfad661a496593dba8df6fdc5e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:00:07.878549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.879208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a7211b7c-9413-4d3b-8823-8ad00987f1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:07.880712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1110/2253] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:00:07 2022] DELETE /identity/v3/projects/8214dfad661a496593dba8df6fdc5e4e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:07.885857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2c5d6-26d3-4482-a4eb-fe81aa03fef4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:07.886166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2c5d6-26d3-4482-a4eb-fe81aa03fef4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:07.886462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2c5d6-26d3-4482-a4eb-fe81aa03fef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:07.906747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-38b2c5d6-26d3-4482-a4eb-fe81aa03fef4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:07.907945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-38b2c5d6-26d3-4482-a4eb-fe81aa03fef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:07.967966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1144/2254] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:07.975852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865752-a73d-4717-8e13-7cb1a3a906de None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:07.976227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865752-a73d-4717-8e13-7cb1a3a906de None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:07.976505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865752-a73d-4717-8e13-7cb1a3a906de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:07.997023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-68865752-a73d-4717-8e13-7cb1a3a906de None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:07.998289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-68865752-a73d-4717-8e13-7cb1a3a906de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:08.006183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1111/2255] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:08.010319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2c5d6-26d3-4482-a4eb-fe81aa03fef4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:08.020398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2c5d6-26d3-4482-a4eb-fe81aa03fef4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:08.025080 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:08.025716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.025990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.026257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.026891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:00:08.027947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:08.029554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:08.040877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1145/2256] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:00:08.045873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68865752-a73d-4717-8e13-7cb1a3a906de None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:08.054924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68865752-a73d-4717-8e13-7cb1a3a906de None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:08.059524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:08.060137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.060411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.060681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.061336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:08.063205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:08.089438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1112/2257] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:00:08 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:00:08.093841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:08.104101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9878-d6d4-4d01-9ca6-feb1f78578c4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:08.166281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:08.166939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.167245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.167520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.168664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:08.170754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:08.193572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:08.202499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1146/2258] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:00:08 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:00:08.206779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:08.217431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ba031-96f5-4058-9cd0-f996319ee8e3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:08.222252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:08.222859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.223133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.223400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.224144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:08.225798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:08.237035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1113/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:00:08.240997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:08.252624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15b0e7-3f94-4266-bfe4-993694e8eff4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:08.257609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569ae1-35c2-42c1-b192-ae2650c87a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:08.258216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569ae1-35c2-42c1-b192-ae2650c87a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.258493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569ae1-35c2-42c1-b192-ae2650c87a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.258766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569ae1-35c2-42c1-b192-ae2650c87a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.259571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569ae1-35c2-42c1-b192-ae2650c87a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:08.261188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569ae1-35c2-42c1-b192-ae2650c87a22 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:08.273090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1147/2260] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:00:08.278712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:08.291791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0ef52-d2d6-49d6-8bec-2215cb36e87b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:08.296425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:08.297174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.297588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.297940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] PATH_INFO: `/v3/projects/eae0b0bfcb0b457aa69cd77f582e5ffc/users/f9c90899d1de4176a959fc4f828aac95/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.319903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae0b0bfcb0b457aa69cd77f582e5ffc, user_id=f9c90899d1de4176a959fc4f828aac95, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:08.323453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:08.344370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1114/2261] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:00:08 2022] PUT /identity/v3/projects/eae0b0bfcb0b457aa69cd77f582e5ffc/users/f9c90899d1de4176a959fc4f828aac95/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:08.355778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3b525-92fd-4529-bc06-8108ac7bdd76 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.356084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3b525-92fd-4529-bc06-8108ac7bdd76 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.356410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3b525-92fd-4529-bc06-8108ac7bdd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.397471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0aa3b525-92fd-4529-bc06-8108ac7bdd76 None None] MFA Rules not processed for user `f9c90899d1de4176a959fc4f828aac95`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:08.399130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0aa3b525-92fd-4529-bc06-8108ac7bdd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:08.470559 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1148/2262] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:08.482258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:08.483961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:08.547213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643f0d7-867c-438c-ae2b-7c4b8974c473 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:08.551973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f1dc5-1e5e-44a8-8af0-3bb0272bbd21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:08.552708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1dc5-1e5e-44a8-8af0-3bb0272bbd21 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:08.552986 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1dc5-1e5e-44a8-8af0-3bb0272bbd21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:08.553278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1dc5-1e5e-44a8-8af0-3bb0272bbd21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:08.564176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f1dc5-1e5e-44a8-8af0-3bb0272bbd21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:08.565903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f1dc5-1e5e-44a8-8af0-3bb0272bbd21 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:08.578960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1115/2263] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:00:18.467521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=38 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:18.573298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:20.154671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d61fd-a53d-4c7a-b07e-40aae4cd8abc None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:20.155154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d61fd-a53d-4c7a-b07e-40aae4cd8abc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:20.155239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d61fd-a53d-4c7a-b07e-40aae4cd8abc None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:20.156659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1149/2264] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 9 15:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 15:00:20.162488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1d002-0f4f-438c-86ca-db3312ee6cd7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:20.163002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1d002-0f4f-438c-86ca-db3312ee6cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:20.163183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1d002-0f4f-438c-86ca-db3312ee6cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:20.186437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-03e1d002-0f4f-438c-86ca-db3312ee6cd7 None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:20.187268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-03e1d002-0f4f-438c-86ca-db3312ee6cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:20.267186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1116/2265] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 9 15:00:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:20.270295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d61fd-a53d-4c7a-b07e-40aae4cd8abc None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:20.297409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d61fd-a53d-4c7a-b07e-40aae4cd8abc None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:20.304248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:20.305161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:20.305670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:20.306131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:20.308932 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:20.324907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:20.327703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:20.342274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1150/2266] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:00:29.542121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba20c4-4239-4fb4-9648-b00bb1d58112 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:29.542620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba20c4-4239-4fb4-9648-b00bb1d58112 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:29.543218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba20c4-4239-4fb4-9648-b00bb1d58112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:29.575858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-44ba20c4-4239-4fb4-9648-b00bb1d58112 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:29.577271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-44ba20c4-4239-4fb4-9648-b00bb1d58112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:29.652192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1117/2267] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 15:00:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:29.675489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:29.676906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:29.759678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d3791-b870-461f-a46e-7e91368b6395 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:29.765068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:29.765803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:29.766289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:29.766652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:29.782403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:29.784624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:29.793420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1151/2268] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:00:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:00:39.641400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=54 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:39.794137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=38 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:41.158501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593da94-2c53-4689-b05c-ba45e64cc300 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:41.158976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593da94-2c53-4689-b05c-ba45e64cc300 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:41.159132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593da94-2c53-4689-b05c-ba45e64cc300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:41.179370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1593da94-2c53-4689-b05c-ba45e64cc300 None None] MFA Rules not processed for user `770105b06e084179a819a54d28b70cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:41.180554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1593da94-2c53-4689-b05c-ba45e64cc300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:41.240003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1118/2269] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:46.145966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:46.206756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43220f6a-e661-4b0d-9ba9-ab23a78bd967 service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:46.211171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:46.211780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:46.212053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:46.212321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] PATH_INFO: `/v3/users/f9c90899d1de4176a959fc4f828aac95` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:46.213463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c90899d1de4176a959fc4f828aac95)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:46.215370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:46.264658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c90899d1de4176a959fc4f828aac95'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:46.271828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c90899d1de4176a959fc4f828aac95'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:46.285724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c90899d1de4176a959fc4f828aac95'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:46.286548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c90899d1de4176a959fc4f828aac95'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:46.296690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c90899d1de4176a959fc4f828aac95'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:46.303678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1152/2270] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:00:46 2022] DELETE /identity/v3/users/f9c90899d1de4176a959fc4f828aac95 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:46.308215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:46.308731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:46.308731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:46.328153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:46.329387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:46.400483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1119/2271] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:00:46.411043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:46.412760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:46.477785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde7296-6fe8-4b39-96cb-fd5174b96c4b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:46.481706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71414f7-0983-4538-a5d5-fa9690f5742d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:46.482307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71414f7-0983-4538-a5d5-fa9690f5742d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:46.482582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71414f7-0983-4538-a5d5-fa9690f5742d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:46.483221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71414f7-0983-4538-a5d5-fa9690f5742d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:46.493575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71414f7-0983-4538-a5d5-fa9690f5742d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:46.495041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71414f7-0983-4538-a5d5-fa9690f5742d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:46.505479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1153/2272] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:00:46.928356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:46.929892 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:46.943507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8841d-0144-41b9-84ad-32793458ba56 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:46.948457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:46.949097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:46.949406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:46.949675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] PATH_INFO: `/v3/projects/eae0b0bfcb0b457aa69cd77f582e5ffc` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:46.950800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae0b0bfcb0b457aa69cd77f582e5ffc)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:46.952706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.001620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae0b0bfcb0b457aa69cd77f582e5ffc', 'request_id': 'req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:47.007646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae0b0bfcb0b457aa69cd77f582e5ffc', 'request_id': 'req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:47.013178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae0b0bfcb0b457aa69cd77f582e5ffc', 'request_id': 'req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:47.028745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] The token cache is being invalidate because project eae0b0bfcb0b457aa69cd77f582e5ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:00:47.029032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:47.029748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1540f4c-ff2a-444d-ad68-2c5b9afa316f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:47.031280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1120/2273] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:00:46 2022] DELETE /identity/v3/projects/eae0b0bfcb0b457aa69cd77f582e5ffc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:00:47.036315 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015a301-f11c-4c28-8beb-b13457acd2e3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.036599 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015a301-f11c-4c28-8beb-b13457acd2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.036872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015a301-f11c-4c28-8beb-b13457acd2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.058975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3015a301-f11c-4c28-8beb-b13457acd2e3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:47.060198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3015a301-f11c-4c28-8beb-b13457acd2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:47.077664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:47.120486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1154/2274] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:47.128415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771fc5-2a65-429f-95cf-378fcff3c1ed None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.128699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771fc5-2a65-429f-95cf-378fcff3c1ed None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.128998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771fc5-2a65-429f-95cf-378fcff3c1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.148621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-46771fc5-2a65-429f-95cf-378fcff3c1ed None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:47.149849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-46771fc5-2a65-429f-95cf-378fcff3c1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:47.158400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1121/2275] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:47.162571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015a301-f11c-4c28-8beb-b13457acd2e3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:47.172777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015a301-f11c-4c28-8beb-b13457acd2e3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:47.177375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:47.177959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.178237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.178508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.179148 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:00:47.179446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:47.181062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.192259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1155/2276] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:00:47.196372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771fc5-2a65-429f-95cf-378fcff3c1ed None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:47.205526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771fc5-2a65-429f-95cf-378fcff3c1ed None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:47.210137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:47.210725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.211002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.211284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.211901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:47.213845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.235812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:47.249245 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1122/2277] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:00:47 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:00:47.253407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:47.263399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd9df-1a85-44e5-96ef-40fd1d13cac8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:47.320508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:47.321102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.321404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.321672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.322282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:47.324238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.344718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1156/2278] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:00:47 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:00:47.348825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:47.359536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df5632-3717-445c-96f0-6370f31ba1cc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:47.364052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:47.364642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.364917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.365188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.365899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:47.367476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.377291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1123/2279] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:00:47.380938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:47.390883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6cfe7-ebfa-442b-9855-7124e09fda9e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:47.395543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14623a1-c597-4a5c-bd5a-7a1f6839ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:47.396130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14623a1-c597-4a5c-bd5a-7a1f6839ebfc admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.396407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14623a1-c597-4a5c-bd5a-7a1f6839ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.396678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14623a1-c597-4a5c-bd5a-7a1f6839ebfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.397382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14623a1-c597-4a5c-bd5a-7a1f6839ebfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:47.398953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14623a1-c597-4a5c-bd5a-7a1f6839ebfc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.409990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1157/2280] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:00:47.414009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:47.424479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5d3f3-593a-42c8-8cde-df79bdce1e33 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:47.429053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:47.429719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.429998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.430270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] PATH_INFO: `/v3/projects/07257eac136c4fa185e8ce702c9ddd81/users/7bd4db059b2d4c17ab19beefc6203214/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.444414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07257eac136c4fa185e8ce702c9ddd81, user_id=7bd4db059b2d4c17ab19beefc6203214, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:47.447078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.459902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1124/2281] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:00:47 2022] PUT /identity/v3/projects/07257eac136c4fa185e8ce702c9ddd81/users/7bd4db059b2d4c17ab19beefc6203214/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:47.470542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc1fc9-0680-4666-a150-124bd71bf236 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.470824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc1fc9-0680-4666-a150-124bd71bf236 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.471094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc1fc9-0680-4666-a150-124bd71bf236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.498989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-13cc1fc9-0680-4666-a150-124bd71bf236 None None] MFA Rules not processed for user `7bd4db059b2d4c17ab19beefc6203214`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:47.500198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-13cc1fc9-0680-4666-a150-124bd71bf236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:47.563577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1158/2282] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:47.574942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:47.577173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:47.632576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2bea-5383-43e5-96a0-49c9fe721e54 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:47.636350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1bcef-f288-44a9-8112-9f67a83c7226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:47.636945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1bcef-f288-44a9-8112-9f67a83c7226 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:47.637242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1bcef-f288-44a9-8112-9f67a83c7226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:47.637519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1bcef-f288-44a9-8112-9f67a83c7226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:47.645984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1bcef-f288-44a9-8112-9f67a83c7226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:47.647388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1bcef-f288-44a9-8112-9f67a83c7226 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:47.656635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1125/2283] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:00:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:00:54.744687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc1fc9-0680-4666-a150-124bd71bf236 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:54.763257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:54.808002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc1fc9-0680-4666-a150-124bd71bf236 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:54.812518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:54.813135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:54.813441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:54.813714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] PATH_INFO: `/v3/users/7bd4db059b2d4c17ab19beefc6203214` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:54.814832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd4db059b2d4c17ab19beefc6203214)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:54.816756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:54.860949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd4db059b2d4c17ab19beefc6203214'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:54.867710 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd4db059b2d4c17ab19beefc6203214'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:54.879120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd4db059b2d4c17ab19beefc6203214'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:54.879774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd4db059b2d4c17ab19beefc6203214'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:54.886690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd4db059b2d4c17ab19beefc6203214'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:54.892719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1159/2284] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:00:54 2022] DELETE /identity/v3/users/7bd4db059b2d4c17ab19beefc6203214 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:54.897159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:54.897698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:54.897797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:54.916950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:54.918182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:54.982813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1126/2285] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:54.989462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:54.990906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:55.055574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646b50-2c81-44c8-b6a0-141b1497da2a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.059253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c6c8f-d1f1-470d-a0c2-8ef771a6bbeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.059856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c6c8f-d1f1-470d-a0c2-8ef771a6bbeb service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.060131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c6c8f-d1f1-470d-a0c2-8ef771a6bbeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.060397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c6c8f-d1f1-470d-a0c2-8ef771a6bbeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.069658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c6c8f-d1f1-470d-a0c2-8ef771a6bbeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.071035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c6c8f-d1f1-470d-a0c2-8ef771a6bbeb service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.081093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1160/2286] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:00:55.445512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:55.447074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:55.462081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72c60-4fcc-43a3-a7bf-60a7df7e4d2c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.466848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.467517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.467758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.468027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] PATH_INFO: `/v3/projects/07257eac136c4fa185e8ce702c9ddd81` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.469292 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07257eac136c4fa185e8ce702c9ddd81)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.471577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.521899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07257eac136c4fa185e8ce702c9ddd81', 'request_id': 'req-cc4e197d-bee1-40cc-8911-431e400e8b95'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:55.528153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07257eac136c4fa185e8ce702c9ddd81', 'request_id': 'req-cc4e197d-bee1-40cc-8911-431e400e8b95'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:55.534028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07257eac136c4fa185e8ce702c9ddd81', 'request_id': 'req-cc4e197d-bee1-40cc-8911-431e400e8b95'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:55.535663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:55.548473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] The token cache is being invalidate because project 07257eac136c4fa185e8ce702c9ddd81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:00:55.548955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:55.549592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc4e197d-bee1-40cc-8911-431e400e8b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:00:55.551096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1127/2287] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:00:55 2022] DELETE /identity/v3/projects/07257eac136c4fa185e8ce702c9ddd81 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:55.556179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa10414-8081-4d7f-938d-bcb4e4ba7e8f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.556533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa10414-8081-4d7f-938d-bcb4e4ba7e8f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.556810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa10414-8081-4d7f-938d-bcb4e4ba7e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.578497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0aa10414-8081-4d7f-938d-bcb4e4ba7e8f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:55.579707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0aa10414-8081-4d7f-938d-bcb4e4ba7e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:55.614715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:00:55.641233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1161/2288] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:55.650779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43068-d1bf-43f0-82f0-3db725d7d1db None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.651067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43068-d1bf-43f0-82f0-3db725d7d1db None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.651344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43068-d1bf-43f0-82f0-3db725d7d1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.672565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5bd43068-d1bf-43f0-82f0-3db725d7d1db None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:55.673824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5bd43068-d1bf-43f0-82f0-3db725d7d1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:55.681787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1128/2289] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:55.686024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa10414-8081-4d7f-938d-bcb4e4ba7e8f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:55.696457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa10414-8081-4d7f-938d-bcb4e4ba7e8f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.701312 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.702509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.702858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.703199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.703855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:00:55.704159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.706679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.718455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1162/2290] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:00:55.722770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43068-d1bf-43f0-82f0-3db725d7d1db None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:55.732639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43068-d1bf-43f0-82f0-3db725d7d1db None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.737488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.738091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.738376 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.738644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.739270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.741170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.772911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1129/2291] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:00:55.774066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:55.784832 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32cbf-be59-4cd2-832f-074eb3030db8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.841184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.841851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.842132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.842402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.843656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.845623 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.866466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1163/2292] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:00:55 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:00:55.870693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:55.881001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ad9eb-da98-4b05-9931-b85089b5c4c0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.885877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.886473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.886750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.887017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.887652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.889311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.903506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1130/2293] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:00:55.904411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:55.915491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac4bd7-baa9-40c4-bf50-49b1a3079a0e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.919640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d094c32-3266-49dc-a8c6-cc4f5c274028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.920233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094c32-3266-49dc-a8c6-cc4f5c274028 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.920512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094c32-3266-49dc-a8c6-cc4f5c274028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.920876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094c32-3266-49dc-a8c6-cc4f5c274028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.921551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d094c32-3266-49dc-a8c6-cc4f5c274028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.923156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d094c32-3266-49dc-a8c6-cc4f5c274028 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.934843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1164/2294] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:00:55.938983 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:55.948932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3473f-6bbb-4152-9ab3-0d68ba4d53dc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:55.954280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:55.954925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.955205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.955503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] PATH_INFO: `/v3/projects/515d1ca3dcd249ada56b21a0a6e3a5b9/users/9a50fbbf18584bb1a784283e768be7a2/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:55.970647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=515d1ca3dcd249ada56b21a0a6e3a5b9, user_id=9a50fbbf18584bb1a784283e768be7a2, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:55.973334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:55.987117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1131/2295] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:00:55 2022] PUT /identity/v3/projects/515d1ca3dcd249ada56b21a0a6e3a5b9/users/9a50fbbf18584bb1a784283e768be7a2/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:00:55.992127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd70101-f248-42fb-a4b5-3a39c1e1620b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:55.992424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd70101-f248-42fb-a4b5-3a39c1e1620b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:55.992797 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd70101-f248-42fb-a4b5-3a39c1e1620b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:56.012179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ebd70101-f248-42fb-a4b5-3a39c1e1620b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:00:56.013503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ebd70101-f248-42fb-a4b5-3a39c1e1620b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:56.079621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1165/2296] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:00:56.089088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:00:56.090814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:00:56.161037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fda7f-b318-408c-b2d3-eac5691bc90b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:00:56.165592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:00:56.166217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:00:56.166501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:00:56.166774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:00:56.178444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:00:56.180032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:00:56.195511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1132/2297] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:01:00.608195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e116f50-cb7c-4a4e-b30c-9520c54a2cd9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:00.608724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e116f50-cb7c-4a4e-b30c-9520c54a2cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:00.609087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e116f50-cb7c-4a4e-b30c-9520c54a2cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:00.642152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3e116f50-cb7c-4a4e-b30c-9520c54a2cd9 None None] MFA Rules not processed for user `9a50fbbf18584bb1a784283e768be7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:01:00.643391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3e116f50-cb7c-4a4e-b30c-9520c54a2cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:00.679023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:00.715615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1166/2298] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:01:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:01:00.720079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:00.730194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632636bb-c31f-4700-bda4-e702dbe0593e service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:00.735343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:00.736080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:00.736569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:00.737045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:00.738414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:00.741327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:00.771958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1133/2299] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:01:00 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:01:00.775653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e116f50-cb7c-4a4e-b30c-9520c54a2cd9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:00.787278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e116f50-cb7c-4a4e-b30c-9520c54a2cd9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:00.845867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:00.846483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:00.846760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:00.847030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:00.847696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:00.849698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:00.873410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1167/2300] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:01:00 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:01:00.877817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:00.888197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb50d29-4719-4019-80df-ef2cfa5713e3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:00.893523 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:00.894135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:00.894410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:00.894678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:00.895310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:00.896899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:00.908303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1134/2301] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:01:00.912447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:00.922999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8a8eb-c9cb-455d-834f-8cee81b78948 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:00.927915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:00.928583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:00.928860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:00.929133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] PATH_INFO: `/v3/projects/a8c71efcec054d2f8ae4ced89127d243/users/6e0bcdb2603849cdbed230b141b7196e/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:00.947124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c71efcec054d2f8ae4ced89127d243, user_id=6e0bcdb2603849cdbed230b141b7196e, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:00.950071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:00.968399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1168/2302] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:01:00 2022] PUT /identity/v3/projects/a8c71efcec054d2f8ae4ced89127d243/users/6e0bcdb2603849cdbed230b141b7196e/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:01:00.975610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:00.990069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c2d1b-e93b-4457-a1e2-f8e646ded47c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:01.056508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:01.065260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd8fb5-ad47-407c-a1cb-0286941c7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:01.066223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8fb5-ad47-407c-a1cb-0286941c7617 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:01.066730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8fb5-ad47-407c-a1cb-0286941c7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:01.067001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8fb5-ad47-407c-a1cb-0286941c7617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:01.067781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd8fb5-ad47-407c-a1cb-0286941c7617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:01.069882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd8fb5-ad47-407c-a1cb-0286941c7617 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:01.080614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1135/2303] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:01:01.084882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:01.096292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa32cd0-fe7b-4c2e-aed3-d48aae805bb1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:01.101885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:01.102560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:01.102841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:01.103113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] PATH_INFO: `/v3/projects/a8c71efcec054d2f8ae4ced89127d243/users/6e0bcdb2603849cdbed230b141b7196e/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:01.105100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c71efcec054d2f8ae4ced89127d243, user_id=6e0bcdb2603849cdbed230b141b7196e, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:01.107813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:01.119554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1169/2304] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:01:01 2022] PUT /identity/v3/projects/a8c71efcec054d2f8ae4ced89127d243/users/6e0bcdb2603849cdbed230b141b7196e/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:01:05.150316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25c0d-05a6-405a-a7bd-082e35f31b3b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:05.150854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25c0d-05a6-405a-a7bd-082e35f31b3b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:05.150854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25c0d-05a6-405a-a7bd-082e35f31b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:05.179551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ade25c0d-05a6-405a-a7bd-082e35f31b3b None None] MFA Rules not processed for user `6e0bcdb2603849cdbed230b141b7196e`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:01:05.180676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ade25c0d-05a6-405a-a7bd-082e35f31b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:05.252942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1136/2305] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:01:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:01:05.270199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:05.272287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:05.349431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a426ba-1f6a-43d9-8ce9-895b7129a62e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:05.354807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10eb9ce-8d2d-4559-9655-6b0a77bdcc23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:05.355409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb9ce-8d2d-4559-9655-6b0a77bdcc23 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:05.355752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb9ce-8d2d-4559-9655-6b0a77bdcc23 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:05.355955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb9ce-8d2d-4559-9655-6b0a77bdcc23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:05.380793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:05.414162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10eb9ce-8d2d-4559-9655-6b0a77bdcc23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:05.415803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10eb9ce-8d2d-4559-9655-6b0a77bdcc23 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:05.425127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1170/2306] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:01:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:01:15.250073 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=37 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:15.420800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=16 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:22.786953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8238d-db53-43c8-b621-9a930a67a5fe None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:22.787435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8238d-db53-43c8-b621-9a930a67a5fe None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:22.787435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8238d-db53-43c8-b621-9a930a67a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:22.808876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7ca8238d-db53-43c8-b621-9a930a67a5fe None None] MFA Rules not processed for user `9a50fbbf18584bb1a784283e768be7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:01:22.810147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7ca8238d-db53-43c8-b621-9a930a67a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:22.817977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1137/2307] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:01:26.644219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:26.644990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:26.644990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:26.667510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] MFA Rules not processed for user `3c0c848f7858486c91773b34047d8aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:01:26.668793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:26.728183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1171/2308] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:01:32.803383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:36.720919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=25 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:37.050756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ee5f-51fd-4d64-90b0-112c3143b465 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:37.051267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ee5f-51fd-4d64-90b0-112c3143b465 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:37.051267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ee5f-51fd-4d64-90b0-112c3143b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:37.070340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c159ee5f-51fd-4d64-90b0-112c3143b465 None None] MFA Rules not processed for user `cd6c40dd492e40599324ef05011895f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:01:37.071551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c159ee5f-51fd-4d64-90b0-112c3143b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:37.126141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1138/2309] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:01:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:01:41.030705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:01:41.032850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:41.106727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274bb87-5902-463a-b580-ec9c3677c964 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:01:41.111546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:01:41.112377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:41.112758 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:41.113094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:41.127482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:01:41.130885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:01:41.144715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1172/2310] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:01:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:01:47.119142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=25 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:51.140098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=26 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:01:54.442253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ae1c-c7ce-4681-ae97-c7dcd6370b45 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:01:54.442772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ae1c-c7ce-4681-ae97-c7dcd6370b45 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:01:54.442885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ae1c-c7ce-4681-ae97-c7dcd6370b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:01:54.462096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0702ae1c-c7ce-4681-ae97-c7dcd6370b45 None None] MFA Rules not processed for user `770105b06e084179a819a54d28b70cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:01:54.463287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0702ae1c-c7ce-4681-ae97-c7dcd6370b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:01:54.517053 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1139/2311] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:02:04.509856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=25 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:19.475265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:19.484068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b6fc9-367d-49ce-a563-8b30776b0bf1 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:19.487692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:19.488273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:19.488546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:19.488813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:19.490692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:19.504500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:19.505949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:19.517345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1173/2312] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:20.706761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ae1c-c7ce-4681-ae97-c7dcd6370b45 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:20.716296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ae1c-c7ce-4681-ae97-c7dcd6370b45 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:20.719814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:20.720405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:20.720680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:20.720953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:20.722701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:20.735904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:20.737323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:20.747775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1140/2313] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:22.955197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:22.956959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:23.011824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef54c08-e157-4df4-b57c-99cbd1ef606e service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:23.015447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:23.016055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:23.016331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:23.016601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:23.020070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:23.031764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:23.033189 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:23.042438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1174/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:25.818851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:25.830369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633efa-5fc1-4565-96ec-2c19d012f232 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:25.837030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c8e36-cbe1-4e7d-8554-c30129e36dc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:25.837661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c8e36-cbe1-4e7d-8554-c30129e36dc9 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:25.837942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c8e36-cbe1-4e7d-8554-c30129e36dc9 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:25.838213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c8e36-cbe1-4e7d-8554-c30129e36dc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:25.839894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-440c8e36-cbe1-4e7d-8554-c30129e36dc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:25.850730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c8e36-cbe1-4e7d-8554-c30129e36dc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:25.852338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c8e36-cbe1-4e7d-8554-c30129e36dc9 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:25.861176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1141/2315] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:31.460627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:31.469700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1c64f-b1f7-4de5-a1df-c1d7e31fa079 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:31.473293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:31.473873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:31.474146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:31.474412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:31.476121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:31.485333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:31.487052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:31.495004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1175/2316] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:35.283824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a69e7f-7d06-4fbd-b35f-c2ca523bb916 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:35.284885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a69e7f-7d06-4fbd-b35f-c2ca523bb916 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:35.285502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a69e7f-7d06-4fbd-b35f-c2ca523bb916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:35.307106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-39a69e7f-7d06-4fbd-b35f-c2ca523bb916 None None] MFA Rules not processed for user `b72887db5ed34dbebe0d5745cf77c4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:35.308325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-39a69e7f-7d06-4fbd-b35f-c2ca523bb916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:35.370566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1142/2317] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:35.874766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:35.883867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc221ae-47ef-4eb8-b551-d477ad4c482b service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:35.887632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b09de6-3d55-4ddf-b211-22b41906f15c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:35.888311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b09de6-3d55-4ddf-b211-22b41906f15c service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:35.888653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b09de6-3d55-4ddf-b211-22b41906f15c service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:35.888984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b09de6-3d55-4ddf-b211-22b41906f15c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:35.890800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-19b09de6-3d55-4ddf-b211-22b41906f15c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:35.900557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b09de6-3d55-4ddf-b211-22b41906f15c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:35.902122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b09de6-3d55-4ddf-b211-22b41906f15c service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:35.911025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1176/2318] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:02:36.034381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a69e7f-7d06-4fbd-b35f-c2ca523bb916 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:36.056381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a69e7f-7d06-4fbd-b35f-c2ca523bb916 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:36.061439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:36.062866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.063344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.063805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.066017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:36.079478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632583f4-7f5a-4770-a5be-db6956043e33 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.080044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632583f4-7f5a-4770-a5be-db6956043e33 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.080341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632583f4-7f5a-4770-a5be-db6956043e33 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.081766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1177/2319] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 15:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 15:02:36.084368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:36.086103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:36.089676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35797d12-97ca-4b91-8afb-53c92961008e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.089960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35797d12-97ca-4b91-8afb-53c92961008e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.090234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35797d12-97ca-4b91-8afb-53c92961008e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.102134 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1143/2320] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:36.114428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-35797d12-97ca-4b91-8afb-53c92961008e None None] MFA Rules not processed for user `f9edd8cf448e42fd8e888c8f531fa70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:36.115769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-35797d12-97ca-4b91-8afb-53c92961008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:36.122332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1178/2321] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 15:02:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:36.138774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:36.151063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f41d-1cbe-4433-b8ed-924ea2c7db52 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:36.154961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f9bd5-2ef7-4086-80df-c0da91aa4033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:36.155568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9bd5-2ef7-4086-80df-c0da91aa4033 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.155904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9bd5-2ef7-4086-80df-c0da91aa4033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.156699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9bd5-2ef7-4086-80df-c0da91aa4033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.164025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f9bd5-2ef7-4086-80df-c0da91aa4033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:36.165640 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f9bd5-2ef7-4086-80df-c0da91aa4033 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:36.173750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1144/2322] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:36.269295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b67cc-f159-4104-a184-973945cdfedd None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.269816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b67cc-f159-4104-a184-973945cdfedd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.270377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b67cc-f159-4104-a184-973945cdfedd None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.272333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1179/2323] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 15:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 15:02:36.278672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.279052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.279420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.300538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:36.302513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:36.315275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1145/2324] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 15:02:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:02:36.332959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b67cc-f159-4104-a184-973945cdfedd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:36.367647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b67cc-f159-4104-a184-973945cdfedd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:36.372674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:36.373486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.375300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.375780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.394084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:36.396159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:36.419534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1180/2325] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:36.485164 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:36.486610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:36.561432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edfaa5-7ee0-48e9-96bc-51c7eb4048ec None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:36.568907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:36.569683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:36.569927 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:36.570193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:36.572268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:36.586036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:36.587914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:36.599851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1146/2326] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:41.931868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:41.933344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:41.948869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=54 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:41.956304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69268fa8-a742-484d-a145-0b3bb2e040cf service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:41.966184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:41.967302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:41.967806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:41.968082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:41.971445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:42.001571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:42.003210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:42.015017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1181/2327] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:46.595452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:46.685574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:46.698187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f6588-2101-4f61-af75-ddde911ff638 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:46.703626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9db3a6-356f-40e2-83aa-118da5a9be5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:46.704220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db3a6-356f-40e2-83aa-118da5a9be5d service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:46.704494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db3a6-356f-40e2-83aa-118da5a9be5d service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:46.704760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db3a6-356f-40e2-83aa-118da5a9be5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:46.706474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8e9db3a6-356f-40e2-83aa-118da5a9be5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:46.718325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9db3a6-356f-40e2-83aa-118da5a9be5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:46.719887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9db3a6-356f-40e2-83aa-118da5a9be5d service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:46.730072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1147/2328] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:02:49.099027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:49.108059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf2b6-8660-4976-8579-2e40c2ad92d5 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:49.111660 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:49.112257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:49.112539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:49.112806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:49.114541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:49.189113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:49.190620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:49.201072 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1182/2329] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:49.823144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fcc6-6ce6-4f63-9790-9ad04387b208 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:49.824237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fcc6-6ce6-4f63-9790-9ad04387b208 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:49.824717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fcc6-6ce6-4f63-9790-9ad04387b208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:49.847350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0e66fcc6-6ce6-4f63-9790-9ad04387b208 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:49.848876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0e66fcc6-6ce6-4f63-9790-9ad04387b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:49.913091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1148/2330] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:49.964884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:49.966884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:50.051937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eedcd3-2f42-4c3d-8b9d-8814605f4e00 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:50.057235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:50.057915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:50.058252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:50.058587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:50.067606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:50.069185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:50.077299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1183/2331] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:02:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:02:51.167256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fcc6-6ce6-4f63-9790-9ad04387b208 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:51.190729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fcc6-6ce6-4f63-9790-9ad04387b208 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:51.206813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:51.207442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:51.207736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:51.207988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:51.211898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:51.233547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:51.235996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:51.252052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1149/2332] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:52.766933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:52.790287 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef31b1-e001-404a-bb15-5eccc174234f service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:52.801673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:52.802442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:52.802724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:52.802996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:52.805243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:52.838377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:52.840009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:52.866166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1184/2333] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:53.936651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:53.939459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:53.967549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef166647-3801-4dec-b3d8-ec1419c7cace service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:53.982155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:53.983551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:53.984059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:53.984482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:53.990138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:54.020114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:54.021689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:54.043956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1150/2334] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:54.078982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:54.089856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:54.126469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c266ea-2c51-4f24-baea-57703c4a43dd service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:54.146275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:54.149447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:54.149962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:54.150395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] PATH_INFO: `/v3/users/cd6c40dd492e40599324ef05011895f3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:54.153278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd6c40dd492e40599324ef05011895f3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:54.158117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:54.232942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd6c40dd492e40599324ef05011895f3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.247818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:54.252320 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd6c40dd492e40599324ef05011895f3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.292764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd6c40dd492e40599324ef05011895f3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.293755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd6c40dd492e40599324ef05011895f3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.304205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0778e7f7-6ccd-455c-8e53-3698582e0d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd6c40dd492e40599324ef05011895f3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.310842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1185/2335] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:02:54 2022] DELETE /identity/v3/users/cd6c40dd492e40599324ef05011895f3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:02:54.533372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:54.553778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cabdc-4910-4baf-804c-6a3f473b31c6 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:54.691200 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:54.692163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:54.692751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:54.693183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] PATH_INFO: `/v3/projects/b8051535f56a4e17a0fa1eeab7c08ce7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:54.694787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8051535f56a4e17a0fa1eeab7c08ce7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:54.696908 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:54.805846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8051535f56a4e17a0fa1eeab7c08ce7', 'request_id': 'req-669334db-5cc8-4c1c-ab94-5b696ceee4ba'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.812551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8051535f56a4e17a0fa1eeab7c08ce7', 'request_id': 'req-669334db-5cc8-4c1c-ab94-5b696ceee4ba'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.818620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8051535f56a4e17a0fa1eeab7c08ce7', 'request_id': 'req-669334db-5cc8-4c1c-ab94-5b696ceee4ba'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.822467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:54.835492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] The token cache is being invalidate because project b8051535f56a4e17a0fa1eeab7c08ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:02:54.836704 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.838025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-669334db-5cc8-4c1c-ab94-5b696ceee4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:54.844106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1151/2336] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:02:54 2022] DELETE /identity/v3/projects/b8051535f56a4e17a0fa1eeab7c08ce7 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:02:54.845964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2adb5-96b8-46d7-96b6-3c5b0605a8f1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:54.847032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2adb5-96b8-46d7-96b6-3c5b0605a8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:54.847394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2adb5-96b8-46d7-96b6-3c5b0605a8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:54.887714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-46f2adb5-96b8-46d7-96b6-3c5b0605a8f1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:54.889220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-46f2adb5-96b8-46d7-96b6-3c5b0605a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:54.978936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1186/2337] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:54.987817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc39eeb-9c3c-48d6-8ae4-dca9b8ceb917 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:54.988789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc39eeb-9c3c-48d6-8ae4-dca9b8ceb917 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:54.989285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc39eeb-9c3c-48d6-8ae4-dca9b8ceb917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.028970 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7dc39eeb-9c3c-48d6-8ae4-dca9b8ceb917 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:55.031260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7dc39eeb-9c3c-48d6-8ae4-dca9b8ceb917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:55.055615 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1152/2338] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:55.060196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2adb5-96b8-46d7-96b6-3c5b0605a8f1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:55.077973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2adb5-96b8-46d7-96b6-3c5b0605a8f1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:55.084962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:55.086496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.087039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.087571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.088441 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:02:55.088839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:55.091054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:55.109240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1187/2339] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:02:55.112521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc39eeb-9c3c-48d6-8ae4-dca9b8ceb917 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:55.124426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc39eeb-9c3c-48d6-8ae4-dca9b8ceb917 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:55.133559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:55.134608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.135037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.135459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.136256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:55.140977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:55.220405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:55.227732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1153/2340] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:02:55 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:02:55.245534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd237445-7546-48cf-b835-8c96ef2ea7ed admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:55.357371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:55.359415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.360216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.360652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.370799 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:55.372905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:55.442276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1188/2341] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:02:55 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:02:55.443463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:55.463021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179b7f0-d796-4e6a-baea-2d16c90f9fb2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:55.478048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:55.479186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.482124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.482675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.484167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:55.488318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:55.535889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1154/2342] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:02:55.537431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:55.552094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f06b-3fff-4328-9148-13099acbdda6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:55.557694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfc4f7-24d1-48b5-ba02-1348ecd2fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:55.558443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfc4f7-24d1-48b5-ba02-1348ecd2fb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.559041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfc4f7-24d1-48b5-ba02-1348ecd2fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.559550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfc4f7-24d1-48b5-ba02-1348ecd2fb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.562847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfc4f7-24d1-48b5-ba02-1348ecd2fb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:55.565728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfc4f7-24d1-48b5-ba02-1348ecd2fb2f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:55.586437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1189/2343] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:02:55.590793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:55.603193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d092e8f-fbcf-4607-ae73-8eb7dec45063 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:55.609482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:55.610284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.610667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.611042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] PATH_INFO: `/v3/projects/638634f2c79b473486cabd8292095f2d/users/9430ca7bb90246ff84d0b9e4f762d953/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.649588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] RBAC: Authorizing `identity:create_grant(project_id=638634f2c79b473486cabd8292095f2d, user_id=9430ca7bb90246ff84d0b9e4f762d953, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:55.654762 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:55.686757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1155/2344] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:02:55 2022] PUT /identity/v3/projects/638634f2c79b473486cabd8292095f2d/users/9430ca7bb90246ff84d0b9e4f762d953/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:02:55.698573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ae418-1470-4846-a6f6-74b4c0192d21 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.698849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ae418-1470-4846-a6f6-74b4c0192d21 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.699120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ae418-1470-4846-a6f6-74b4c0192d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.737922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b36ae418-1470-4846-a6f6-74b4c0192d21 None None] MFA Rules not processed for user `9430ca7bb90246ff84d0b9e4f762d953`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:55.739703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b36ae418-1470-4846-a6f6-74b4c0192d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:55.774341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:55.828737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1190/2345] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:02:55.840288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:55.842318 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:55.961374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b16bf9-4ee7-45b2-b39e-cc4529d1fcda admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:55.971689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e03a33-27b8-4215-a5b4-d7e7f96954c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:55.975342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03a33-27b8-4215-a5b4-d7e7f96954c3 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:55.975342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03a33-27b8-4215-a5b4-d7e7f96954c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:55.975342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03a33-27b8-4215-a5b4-d7e7f96954c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:55.989002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e03a33-27b8-4215-a5b4-d7e7f96954c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:55.991182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e03a33-27b8-4215-a5b4-d7e7f96954c3 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:56.003510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1156/2346] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:56.379193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ae418-1470-4846-a6f6-74b4c0192d21 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:56.551945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ae418-1470-4846-a6f6-74b4c0192d21 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:56.557294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:56.558684 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:56.559154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:56.559607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] PATH_INFO: `/v3/users/9430ca7bb90246ff84d0b9e4f762d953` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:56.561523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9430ca7bb90246ff84d0b9e4f762d953)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:56.563784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:56.665166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9430ca7bb90246ff84d0b9e4f762d953'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:56.673872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9430ca7bb90246ff84d0b9e4f762d953'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:56.691275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9430ca7bb90246ff84d0b9e4f762d953'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:56.692948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9430ca7bb90246ff84d0b9e4f762d953'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:56.715462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9430ca7bb90246ff84d0b9e4f762d953'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:56.745542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1191/2347] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:02:56 2022] DELETE /identity/v3/users/9430ca7bb90246ff84d0b9e4f762d953 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:02:56.747306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:56.747661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:56.747973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:56.777016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:56.778560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:56.862644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1157/2348] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:02:56.872574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:56.874503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:56.948650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:56.958199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402358b0-ea40-4fd4-ba4a-d84df51b2e7f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:56.969745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7e424-29ec-478d-8114-e2831244408f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:56.971038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7e424-29ec-478d-8114-e2831244408f service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:56.971038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7e424-29ec-478d-8114-e2831244408f service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:56.971038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7e424-29ec-478d-8114-e2831244408f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:56.992069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7e424-29ec-478d-8114-e2831244408f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:56.995722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7e424-29ec-478d-8114-e2831244408f service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:57.008598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1192/2349] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:02:57.427399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:57.428883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:57.443951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f77de-6474-42fb-a2ee-5c507abc8e8d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:57.460613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:57.462321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:57.462772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:57.463205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] PATH_INFO: `/v3/projects/638634f2c79b473486cabd8292095f2d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:57.464727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=638634f2c79b473486cabd8292095f2d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:57.467843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:57.495857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:57.582728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '638634f2c79b473486cabd8292095f2d', 'request_id': 'req-00a88495-57ac-4538-a485-5c617f206a7d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:57.597448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '638634f2c79b473486cabd8292095f2d', 'request_id': 'req-00a88495-57ac-4538-a485-5c617f206a7d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:57.605595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '638634f2c79b473486cabd8292095f2d', 'request_id': 'req-00a88495-57ac-4538-a485-5c617f206a7d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:57.627404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] The token cache is being invalidate because project 638634f2c79b473486cabd8292095f2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:02:57.627975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:57.628639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-00a88495-57ac-4538-a485-5c617f206a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:02:57.634290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1158/2350] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:02:57 2022] DELETE /identity/v3/projects/638634f2c79b473486cabd8292095f2d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:02:57.635537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e5b44-0497-4bdf-8468-955a046e7614 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:57.637519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e5b44-0497-4bdf-8468-955a046e7614 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:57.638171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e5b44-0497-4bdf-8468-955a046e7614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:57.661432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-378e5b44-0497-4bdf-8468-955a046e7614 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:57.663031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-378e5b44-0497-4bdf-8468-955a046e7614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:57.746581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1193/2351] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:57.752485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e481c-5559-4158-be51-2c6c853f5042 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:57.753141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e481c-5559-4158-be51-2c6c853f5042 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:57.753141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e481c-5559-4158-be51-2c6c853f5042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:57.775265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-513e481c-5559-4158-be51-2c6c853f5042 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:57.776825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-513e481c-5559-4158-be51-2c6c853f5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:57.788765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1159/2352] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:57.794071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e5b44-0497-4bdf-8468-955a046e7614 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:57.811639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e5b44-0497-4bdf-8468-955a046e7614 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:57.824915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:57.825689 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:57.826049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:57.826393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:57.827145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:02:57.827577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:57.829619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:57.849246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1194/2353] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:02:57.852735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e481c-5559-4158-be51-2c6c853f5042 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:57.863151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e481c-5559-4158-be51-2c6c853f5042 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:57.872969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:57.873734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:57.874078 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:57.874456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:57.875180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:57.877185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:57.923884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1160/2354] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:02:57 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:02:57.927457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:57.938990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352dad-673c-4209-bff4-1d1611572773 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.030233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.033420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.033805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.034187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.035099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.037353 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.065245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1195/2355] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:02:57 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:02:58.069256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.083355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2230f9-d0b7-47e7-af49-1bbf3d3f6af2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.089922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.091056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.091531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.092086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.093074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.095097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.107237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1161/2356] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:02:58.111845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.123451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b292e-90af-4d54-850c-fea6f4dae81d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.129266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb28286-fec8-4a62-b76e-5ac84bc8e18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.130075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28286-fec8-4a62-b76e-5ac84bc8e18d admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.130385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28286-fec8-4a62-b76e-5ac84bc8e18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.130663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28286-fec8-4a62-b76e-5ac84bc8e18d admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.131482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28286-fec8-4a62-b76e-5ac84bc8e18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.133094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28286-fec8-4a62-b76e-5ac84bc8e18d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.144707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1196/2357] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:02:58.148930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.159483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6055c-3bc8-49d4-be5d-3e35589163a2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.168034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.168445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.168834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.169230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] PATH_INFO: `/v3/projects/adfe6967086941fb84a3f161b5e4d22c/users/8949b50e8cb6424182aa97c8a171bd5c/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.185574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfe6967086941fb84a3f161b5e4d22c, user_id=8949b50e8cb6424182aa97c8a171bd5c, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.191401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.207402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1162/2358] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:02:58 2022] PUT /identity/v3/projects/adfe6967086941fb84a3f161b5e4d22c/users/8949b50e8cb6424182aa97c8a171bd5c/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:02:58.219964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac42e0-e751-49cd-b5c0-ae9d07a3911a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.220251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac42e0-e751-49cd-b5c0-ae9d07a3911a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.220551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac42e0-e751-49cd-b5c0-ae9d07a3911a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.250537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cbac42e0-e751-49cd-b5c0-ae9d07a3911a None None] MFA Rules not processed for user `8949b50e8cb6424182aa97c8a171bd5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:58.251767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cbac42e0-e751-49cd-b5c0-ae9d07a3911a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:58.309371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:58.339039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1197/2359] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:58.344235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.429007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fac825-a24a-42e2-aec1-67cd87801ef6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.438353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.439057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.439414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.439764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.440467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.442496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.475644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1163/2360] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:02:58 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:02:58.480216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac42e0-e751-49cd-b5c0-ae9d07a3911a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.493396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac42e0-e751-49cd-b5c0-ae9d07a3911a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.567883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.569249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.569654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.571517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.572310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.574758 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.615596 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1198/2361] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:02:58 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:02:58.617014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.630102 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572cbb3-036b-4ad5-8d66-854a4a6f3299 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.643098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.643813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.644298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.644651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.645408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.647302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.679360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1164/2362] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:02:58.683680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.694645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367a015-ae6e-47ab-b216-d9a10e9ba49e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.701266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.701266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.701266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.701266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] PATH_INFO: `/v3/projects/1f17737418624a9bacbc498a8e3e8b9a/users/6f85b410142a4efc99615aeb88d00527/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.720314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f17737418624a9bacbc498a8e3e8b9a, user_id=6f85b410142a4efc99615aeb88d00527, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.723218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.739355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1199/2363] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:02:58 2022] PUT /identity/v3/projects/1f17737418624a9bacbc498a8e3e8b9a/users/6f85b410142a4efc99615aeb88d00527/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:02:58.743624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.754215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6178a5-949b-435d-8af2-b57057ce0a90 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.763434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:02:58.841272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12b32c-8858-4053-ba91-b20cf8447ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.841858 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12b32c-8858-4053-ba91-b20cf8447ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.842190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12b32c-8858-4053-ba91-b20cf8447ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.842494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12b32c-8858-4053-ba91-b20cf8447ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.844551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12b32c-8858-4053-ba91-b20cf8447ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.850279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12b32c-8858-4053-ba91-b20cf8447ac9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.866730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1165/2364] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:02:58.871028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:58.881551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ab409-8296-4ab8-a098-09235f5b32df admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:58.886256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:58.886926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.887204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.887905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] PATH_INFO: `/v3/projects/1f17737418624a9bacbc498a8e3e8b9a/users/6f85b410142a4efc99615aeb88d00527/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.890428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f17737418624a9bacbc498a8e3e8b9a, user_id=6f85b410142a4efc99615aeb88d00527, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:58.893283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:58.911202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1200/2365] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:02:58 2022] PUT /identity/v3/projects/1f17737418624a9bacbc498a8e3e8b9a/users/6f85b410142a4efc99615aeb88d00527/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:02:58.923493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c822-ce30-411e-942d-0955b75519c2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:58.923775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c822-ce30-411e-942d-0955b75519c2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:58.924045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c822-ce30-411e-942d-0955b75519c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:58.955978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9410c822-ce30-411e-942d-0955b75519c2 None None] MFA Rules not processed for user `6f85b410142a4efc99615aeb88d00527`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:02:58.957356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9410c822-ce30-411e-942d-0955b75519c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:59.037241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1166/2366] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:02:59.050370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:59.052167 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:02:59.122859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f90e5-5a61-43c5-920c-87611dfbd56b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:59.129793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b77f2d-445a-402e-9567-eac786928d44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:59.130507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77f2d-445a-402e-9567-eac786928d44 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:59.130781 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77f2d-445a-402e-9567-eac786928d44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:59.131048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77f2d-445a-402e-9567-eac786928d44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:59.141873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b77f2d-445a-402e-9567-eac786928d44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:59.143412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b77f2d-445a-402e-9567-eac786928d44 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:59.154140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1201/2367] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:02:59.244777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c822-ce30-411e-942d-0955b75519c2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:02:59.260797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c822-ce30-411e-942d-0955b75519c2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:02:59.264727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00928142-54cd-4707-8e7e-a1885710c4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:02:59.265967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928142-54cd-4707-8e7e-a1885710c4fe service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:02:59.266242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928142-54cd-4707-8e7e-a1885710c4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:02:59.266509 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928142-54cd-4707-8e7e-a1885710c4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:02:59.336023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00928142-54cd-4707-8e7e-a1885710c4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:02:59.337709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00928142-54cd-4707-8e7e-a1885710c4fe service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:02:59.358307 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1167/2368] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:02:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:09.149709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:09.343699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=86 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:11.896267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca9a61-37e1-4fc0-a8b1-083dfb3585bd None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:11.896845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca9a61-37e1-4fc0-a8b1-083dfb3585bd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:11.896845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca9a61-37e1-4fc0-a8b1-083dfb3585bd None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:11.898225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1202/2369] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 15:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 15:03:11.903170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75faab-1ab7-4a4e-8b52-ec4ed45789cb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:11.903645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75faab-1ab7-4a4e-8b52-ec4ed45789cb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:11.903884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75faab-1ab7-4a4e-8b52-ec4ed45789cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:11.928769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6d75faab-1ab7-4a4e-8b52-ec4ed45789cb None None] MFA Rules not processed for user `7b6809bf91d244e4bed5b8a256a3f95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:11.930066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6d75faab-1ab7-4a4e-8b52-ec4ed45789cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:12.003088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1168/2370] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 15:03:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:12.017997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca9a61-37e1-4fc0-a8b1-083dfb3585bd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:12.020422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b7ca9a61-37e1-4fc0-a8b1-083dfb3585bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:12.080671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca9a61-37e1-4fc0-a8b1-083dfb3585bd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:12.084130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:12.084719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:12.084993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:12.085292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:12.095537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:12.096948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:12.107573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1203/2371] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:14.347961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:14.348446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:14.348554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:14.368096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:14.369289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:14.423977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1169/2372] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 15:03:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:14.436326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:14.437752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:14.511915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaf692-468c-4859-ac5e-4ab42000ff01 service glance] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:14.517378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783e65-2d0a-48e5-b420-63a33a4bef5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:14.517975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783e65-2d0a-48e5-b420-63a33a4bef5c service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:14.518249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783e65-2d0a-48e5-b420-63a33a4bef5c service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:14.518518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783e65-2d0a-48e5-b420-63a33a4bef5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:14.527411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783e65-2d0a-48e5-b420-63a33a4bef5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:14.528995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783e65-2d0a-48e5-b420-63a33a4bef5c service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:14.538191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1204/2373] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:03:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 15:03:17.349622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:17.351185 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:17.425638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533b227-d4c0-4e33-996b-cb5641f3352d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:17.429600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f9597-5ef9-4065-9744-f60607a78583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:17.430216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f9597-5ef9-4065-9744-f60607a78583 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:17.430497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f9597-5ef9-4065-9744-f60607a78583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:17.430826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f9597-5ef9-4065-9744-f60607a78583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:17.432550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9b2f9597-5ef9-4065-9744-f60607a78583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:17.482424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:17.497212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f9597-5ef9-4065-9744-f60607a78583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:17.498661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f9597-5ef9-4065-9744-f60607a78583 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:17.508886 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1170/2374] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:03:21.710675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e292a-a120-417b-bd20-ba3b6d3cb7ef None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:21.711233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e292a-a120-417b-bd20-ba3b6d3cb7ef None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:21.711417 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e292a-a120-417b-bd20-ba3b6d3cb7ef None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:21.713018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1205/2375] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 15:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 15:03:21.722617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:21.723121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:21.723226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:21.746059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] MFA Rules not processed for user `57cca115ce044ff5aacd5ea1a8834402`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:21.747278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:21.783854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e292a-a120-417b-bd20-ba3b6d3cb7ef None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:21.785645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-597e292a-a120-417b-bd20-ba3b6d3cb7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:21.811575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e292a-a120-417b-bd20-ba3b6d3cb7ef None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:21.816779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:21.817429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:21.817707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:21.817979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] PATH_INFO: `/v3/users/770105b06e084179a819a54d28b70cdf` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:21.819112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770105b06e084179a819a54d28b70cdf)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:21.821179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:21.849268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1171/2376] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 15:03:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:21.862303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:21.863689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:21.875908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770105b06e084179a819a54d28b70cdf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:21.877566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ab62a-f009-4e15-85f6-acca5cba64bb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:21.882905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:21.883274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770105b06e084179a819a54d28b70cdf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:21.883510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:21.883786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:21.884057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:21.892549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:21.894227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:21.895994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770105b06e084179a819a54d28b70cdf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:21.896726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770105b06e084179a819a54d28b70cdf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:21.904012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-99d485d0-282f-4b11-840c-d61964e01f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770105b06e084179a819a54d28b70cdf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:21.904695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1172/2377] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:03:21.910539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1206/2378] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:21 2022] DELETE /identity/v3/users/770105b06e084179a819a54d28b70cdf => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:22.223596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:22.238204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acc148-0e66-4c89-bf42-cf8a44ee7978 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:22.318887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:22.319496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.319854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.320143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] PATH_INFO: `/v3/projects/063bdf7448ee45939287687b0e65dd97` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.321541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063bdf7448ee45939287687b0e65dd97)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:22.323755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:22.413630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063bdf7448ee45939287687b0e65dd97', 'request_id': 'req-7b2c55dc-4368-4617-a40e-72c7698b17e4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:22.424479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063bdf7448ee45939287687b0e65dd97', 'request_id': 'req-7b2c55dc-4368-4617-a40e-72c7698b17e4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:22.431355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063bdf7448ee45939287687b0e65dd97', 'request_id': 'req-7b2c55dc-4368-4617-a40e-72c7698b17e4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:22.446191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:22.451148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] The token cache is being invalidate because project 063bdf7448ee45939287687b0e65dd97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:22.451928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:22.452979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7b2c55dc-4368-4617-a40e-72c7698b17e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:22.455909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1173/2379] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:22 2022] DELETE /identity/v3/projects/063bdf7448ee45939287687b0e65dd97 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:22.461092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a86028-f3c0-48bf-8412-42a1161f1b4c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.461587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a86028-f3c0-48bf-8412-42a1161f1b4c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.461945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a86028-f3c0-48bf-8412-42a1161f1b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.472274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:22.487010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a6a86028-f3c0-48bf-8412-42a1161f1b4c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:22.488360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a6a86028-f3c0-48bf-8412-42a1161f1b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:22.557738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1207/2380] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:22.566646 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72acae-ec95-4642-a106-42e9edfd0600 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.566935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72acae-ec95-4642-a106-42e9edfd0600 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.567207 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72acae-ec95-4642-a106-42e9edfd0600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.588854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ba72acae-ec95-4642-a106-42e9edfd0600 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:22.590048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ba72acae-ec95-4642-a106-42e9edfd0600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:22.603937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1174/2381] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:22.604765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a86028-f3c0-48bf-8412-42a1161f1b4c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:22.615711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a86028-f3c0-48bf-8412-42a1161f1b4c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:22.620482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:22.621109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.621421 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.621696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.622377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:22.622681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:22.624316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:22.637411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1208/2382] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 15:03:22.642644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72acae-ec95-4642-a106-42e9edfd0600 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:22.652075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72acae-ec95-4642-a106-42e9edfd0600 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:22.657698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:22.658315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.658598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.658871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.659513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:22.661713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:22.701498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1175/2383] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:22 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:22.705845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:22.716376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108a725-9b8d-4813-b221-c9b2f2d14b02 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:22.787248 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:22.787861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.788143 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.788416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.789060 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:22.791025 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:22.816202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1209/2384] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:22 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:22.821259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:22.835248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe8030-cd2e-42e7-ae25-56d8bfa77964 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:22.840407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:22.841297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.842057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.842348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.843052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:22.845728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:22.863203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1176/2385] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:22.863632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:22.876232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663a675-fe4c-4dcf-9a0e-059d994c068d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:22.881416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5c03f-6f5d-4e13-a2a3-985b82c07efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:22.882345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5c03f-6f5d-4e13-a2a3-985b82c07efc admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.882683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5c03f-6f5d-4e13-a2a3-985b82c07efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.883433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5c03f-6f5d-4e13-a2a3-985b82c07efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.884244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5c03f-6f5d-4e13-a2a3-985b82c07efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:22.885963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5c03f-6f5d-4e13-a2a3-985b82c07efc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:22.898719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1210/2386] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:03:22.902390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:22.925100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a269-8173-4e08-9e0a-22f4563b81f0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:22.929799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:22.930461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.930740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.931016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] PATH_INFO: `/v3/projects/9f3fc73cb24c4a6185a3c9a2b6344cdb/users/54e08ca9e89a40dc935343176fe6aa1b/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:22.948650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3fc73cb24c4a6185a3c9a2b6344cdb, user_id=54e08ca9e89a40dc935343176fe6aa1b, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:22.951628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:22.973477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1177/2387] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:22 2022] PUT /identity/v3/projects/9f3fc73cb24c4a6185a3c9a2b6344cdb/users/54e08ca9e89a40dc935343176fe6aa1b/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:22.983548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c04bfa-c10b-4a1b-a3cd-44518113712a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:22.983833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c04bfa-c10b-4a1b-a3cd-44518113712a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:22.984136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c04bfa-c10b-4a1b-a3cd-44518113712a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.028977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-81c04bfa-c10b-4a1b-a3cd-44518113712a None None] MFA Rules not processed for user `54e08ca9e89a40dc935343176fe6aa1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:23.031007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-81c04bfa-c10b-4a1b-a3cd-44518113712a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:23.099333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:23.104529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1211/2388] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:23.109017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:23.186964 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770a9fe-63ff-4ce6-a058-925e34d567f2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:23.192263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:23.192871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.193215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.193520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.194153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:23.196029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:23.230400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1178/2389] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:23 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:23.234324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c04bfa-c10b-4a1b-a3cd-44518113712a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:23.251303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c04bfa-c10b-4a1b-a3cd-44518113712a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:23.316199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:23.317230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.317230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.317461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.318003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:23.319947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:23.345374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1212/2390] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:23 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:23.349461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:23.360169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c605d8-d5a1-4ee1-a6bb-fc3f73f4a052 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:23.364835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:23.365503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.365779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.366047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.366677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:23.368254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:23.379622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1179/2391] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:23.383921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:23.396284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197245a2-66a2-4dd9-8172-dd6c48519ac8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:23.401115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:23.401891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.402235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.402568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] PATH_INFO: `/v3/projects/f032018959b34950a061c7c3c0b3dbab/users/fc8db4bd11dc4d25b4cc5e704429ebc4/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.418852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f032018959b34950a061c7c3c0b3dbab, user_id=fc8db4bd11dc4d25b4cc5e704429ebc4, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:23.421516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:23.434587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1213/2392] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:23 2022] PUT /identity/v3/projects/f032018959b34950a061c7c3c0b3dbab/users/fc8db4bd11dc4d25b4cc5e704429ebc4/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:23.439412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:23.450643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a24ba2-807b-47e6-9692-8d46e4ad439f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:23.509138 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278af88-7681-47be-8bba-895eb2326c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:23.509775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278af88-7681-47be-8bba-895eb2326c6b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.510055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278af88-7681-47be-8bba-895eb2326c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.510327 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278af88-7681-47be-8bba-895eb2326c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.510958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278af88-7681-47be-8bba-895eb2326c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:23.512533 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278af88-7681-47be-8bba-895eb2326c6b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:23.521959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1180/2393] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:23.526083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:23.536502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4054b-4eff-415f-8694-57377f85ae42 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:23.541036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:23.541692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.541996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.542263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] PATH_INFO: `/v3/projects/f032018959b34950a061c7c3c0b3dbab/users/fc8db4bd11dc4d25b4cc5e704429ebc4/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.544122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f032018959b34950a061c7c3c0b3dbab, user_id=fc8db4bd11dc4d25b4cc5e704429ebc4, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:23.547265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:23.557104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1214/2394] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:23 2022] PUT /identity/v3/projects/f032018959b34950a061c7c3c0b3dbab/users/fc8db4bd11dc4d25b4cc5e704429ebc4/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:23.568935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcd30b-6023-4c3f-843f-ef6824c18ad9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.569244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcd30b-6023-4c3f-843f-ef6824c18ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.569524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcd30b-6023-4c3f-843f-ef6824c18ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.583204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:23.597618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0dfcd30b-6023-4c3f-843f-ef6824c18ad9 None None] MFA Rules not processed for user `fc8db4bd11dc4d25b4cc5e704429ebc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:23.598829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0dfcd30b-6023-4c3f-843f-ef6824c18ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:23.666089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1181/2395] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:23.677829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:23.679638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:23.738796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23713-d3a6-4af2-9468-2c951d7334bb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:23.742408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c215a8-ef07-4b29-9585-3cf36fcc6b96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:23.743009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c215a8-ef07-4b29-9585-3cf36fcc6b96 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:23.743284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c215a8-ef07-4b29-9585-3cf36fcc6b96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:23.743557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c215a8-ef07-4b29-9585-3cf36fcc6b96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:23.752360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c215a8-ef07-4b29-9585-3cf36fcc6b96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:23.753834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c215a8-ef07-4b29-9585-3cf36fcc6b96 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:23.763054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1215/2396] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:24.158613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcd30b-6023-4c3f-843f-ef6824c18ad9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:24.219502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcd30b-6023-4c3f-843f-ef6824c18ad9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:24.223939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:24.224551 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:24.224827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:24.225089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] PATH_INFO: `/v3/users/54e08ca9e89a40dc935343176fe6aa1b` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:24.226217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e08ca9e89a40dc935343176fe6aa1b)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:24.228108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:24.276057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e08ca9e89a40dc935343176fe6aa1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.288682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e08ca9e89a40dc935343176fe6aa1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.295397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e08ca9e89a40dc935343176fe6aa1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.302768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e08ca9e89a40dc935343176fe6aa1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.303375 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e08ca9e89a40dc935343176fe6aa1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.309707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1182/2397] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:24 2022] DELETE /identity/v3/users/54e08ca9e89a40dc935343176fe6aa1b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:24.314344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:24.314631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:24.314934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:24.334016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:24.335208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:24.381724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:24.394314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1216/2398] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:24.403940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:24.405607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:24.419641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:24.466836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4c49-d0e0-48da-ad8c-ad9f8bfe98bd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:24.471370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:24.471918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:24.472197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:24.472469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:24.481918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:24.483343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:24.493086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1183/2399] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:24.757159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:24.759026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:24.772707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d627e-a094-4eae-852d-4c651b572318 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:24.777254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:24.777906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:24.778177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:24.778443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] PATH_INFO: `/v3/projects/9f3fc73cb24c4a6185a3c9a2b6344cdb` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:24.779549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3fc73cb24c4a6185a3c9a2b6344cdb)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:24.781750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:24.826310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3fc73cb24c4a6185a3c9a2b6344cdb', 'request_id': 'req-01f2dcd4-baf4-44b0-872f-12d773f91126'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.832981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3fc73cb24c4a6185a3c9a2b6344cdb', 'request_id': 'req-01f2dcd4-baf4-44b0-872f-12d773f91126'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.838481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3fc73cb24c4a6185a3c9a2b6344cdb', 'request_id': 'req-01f2dcd4-baf4-44b0-872f-12d773f91126'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.850456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] The token cache is being invalidate because project 9f3fc73cb24c4a6185a3c9a2b6344cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:24.850745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.851361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.852971 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1217/2400] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:24 2022] DELETE /identity/v3/projects/9f3fc73cb24c4a6185a3c9a2b6344cdb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:24.856990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:24.866876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923542-61d0-470d-b84c-6000595f4348 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:24.921280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:24.922562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:24.922872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:24.923156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] PATH_INFO: `/v3/users/fc8db4bd11dc4d25b4cc5e704429ebc4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:24.924305 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc8db4bd11dc4d25b4cc5e704429ebc4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:24.926266 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:24.971845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc8db4bd11dc4d25b4cc5e704429ebc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.983429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc8db4bd11dc4d25b4cc5e704429ebc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.991063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc8db4bd11dc4d25b4cc5e704429ebc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.997402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc8db4bd11dc4d25b4cc5e704429ebc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:24.998059 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9e172b85-76be-4afa-9f14-240ac316c623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc8db4bd11dc4d25b4cc5e704429ebc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:25.004467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1184/2401] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:24 2022] DELETE /identity/v3/users/fc8db4bd11dc4d25b4cc5e704429ebc4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:25.268229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:25.269733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:25.283218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2dcd4-baf4-44b0-872f-12d773f91126 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:25.341520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:25.342160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.342439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.342736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] PATH_INFO: `/v3/projects/f032018959b34950a061c7c3c0b3dbab` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.344341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f032018959b34950a061c7c3c0b3dbab)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:25.346314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:25.390680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f032018959b34950a061c7c3c0b3dbab', 'request_id': 'req-c22ab1ba-355a-4e7a-977e-d27f901bf775'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:25.396643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f032018959b34950a061c7c3c0b3dbab', 'request_id': 'req-c22ab1ba-355a-4e7a-977e-d27f901bf775'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:25.402071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f032018959b34950a061c7c3c0b3dbab', 'request_id': 'req-c22ab1ba-355a-4e7a-977e-d27f901bf775'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:25.414742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] The token cache is being invalidate because project f032018959b34950a061c7c3c0b3dbab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:25.415026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:25.415665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22ab1ba-355a-4e7a-977e-d27f901bf775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:25.417105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1218/2402] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:25 2022] DELETE /identity/v3/projects/f032018959b34950a061c7c3c0b3dbab => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:25.422688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e2eb-a3a5-4dc1-ba53-1df376961761 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.423214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e2eb-a3a5-4dc1-ba53-1df376961761 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.423329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e2eb-a3a5-4dc1-ba53-1df376961761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.442485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5fb4e2eb-a3a5-4dc1-ba53-1df376961761 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:25.443682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5fb4e2eb-a3a5-4dc1-ba53-1df376961761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:25.473326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:25.505480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1185/2403] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:25.513130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c20a4-37db-4322-8a62-75094e4bb150 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.513452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c20a4-37db-4322-8a62-75094e4bb150 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.513751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c20a4-37db-4322-8a62-75094e4bb150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.533711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6a4c20a4-37db-4322-8a62-75094e4bb150 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:25.534906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a4c20a4-37db-4322-8a62-75094e4bb150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:25.542711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1219/2404] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:25.547432 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e2eb-a3a5-4dc1-ba53-1df376961761 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:25.557601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e2eb-a3a5-4dc1-ba53-1df376961761 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:25.562229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:25.562825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.563103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.563374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.564010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:25.564310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:25.565939 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:25.577231 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1186/2405] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:25.581468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c20a4-37db-4322-8a62-75094e4bb150 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:25.586552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:25.592836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c20a4-37db-4322-8a62-75094e4bb150 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:25.598775 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:25.599384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.599653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.599918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.600551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:25.602570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:25.630296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1220/2406] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:25 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:25.634802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:25.644997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc10149-7d40-48a0-9c22-469887944672 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:25.704410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:25.705016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.705450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.705724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.706355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:25.708371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:25.731154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1187/2407] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:25 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:25.735492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:25.745716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97ced5-7ebb-4c52-851f-4c53f8c7db15 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:25.750835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:25.751460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.751951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.752120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.752757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:25.754370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:25.768068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1221/2408] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:25.770442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:25.786304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cac38-1dd3-4951-bbd4-0cc4800d30fd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:25.791086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995e509-2c0e-49ed-97c4-3c986e7f8c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:25.791682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995e509-2c0e-49ed-97c4-3c986e7f8c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.791962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995e509-2c0e-49ed-97c4-3c986e7f8c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.792295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995e509-2c0e-49ed-97c4-3c986e7f8c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.792922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995e509-2c0e-49ed-97c4-3c986e7f8c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:25.794517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995e509-2c0e-49ed-97c4-3c986e7f8c1a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:25.805056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1188/2409] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:25.809104 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:25.819019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ab83-534f-4983-a789-213194425c4f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:25.823662 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:25.824297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.824575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.824847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] PATH_INFO: `/v3/projects/2db8818db6014bf485e0fcc37c6e0e5e/users/e3cd6f53d1b748008caf3d723172be63/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.839492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db8818db6014bf485e0fcc37c6e0e5e, user_id=e3cd6f53d1b748008caf3d723172be63, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:25.842184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:25.856073 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1222/2410] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:25 2022] PUT /identity/v3/projects/2db8818db6014bf485e0fcc37c6e0e5e/users/e3cd6f53d1b748008caf3d723172be63/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:25.866310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7414d92-2001-4aea-88a5-f8649792c222 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:25.866594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7414d92-2001-4aea-88a5-f8649792c222 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:25.866866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7414d92-2001-4aea-88a5-f8649792c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:25.894304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b7414d92-2001-4aea-88a5-f8649792c222 None None] MFA Rules not processed for user `e3cd6f53d1b748008caf3d723172be63`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:25.895526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b7414d92-2001-4aea-88a5-f8649792c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:25.958624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1189/2411] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:25.963211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.027760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b71e1d-e3ce-4926-a4ea-8e76e899f440 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.032601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.033208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.033510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.033764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.034400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.036276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.062170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1223/2412] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:25 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:26.066669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7414d92-2001-4aea-88a5-f8649792c222 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.077025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7414d92-2001-4aea-88a5-f8649792c222 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.102776 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:26.133945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.134542 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.134818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.135088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.135709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.137645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.161146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1190/2413] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:26 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:26.165100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.177711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed504d05-8ff8-444e-a94d-b25687235658 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.182470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.183063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.183360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.183616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.184254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.185964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.201633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1224/2414] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:26.206487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.216538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4444cf-407c-4d91-a74c-2dee8b51bb44 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.221293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.221946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.222226 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.222499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] PATH_INFO: `/v3/projects/57e3c5e98c9c4396a815162c114b0aa2/users/f7370e519994420588e61fdcddea6ccf/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.237227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e3c5e98c9c4396a815162c114b0aa2, user_id=f7370e519994420588e61fdcddea6ccf, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.239910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.253384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1191/2415] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:26 2022] PUT /identity/v3/projects/57e3c5e98c9c4396a815162c114b0aa2/users/f7370e519994420588e61fdcddea6ccf/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:26.257561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.268150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d845-6694-4e2b-870e-209c00a32f2b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.323418 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f189d79-19a8-4f7b-b375-301d042a8229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.324018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f189d79-19a8-4f7b-b375-301d042a8229 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.324298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f189d79-19a8-4f7b-b375-301d042a8229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.324572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f189d79-19a8-4f7b-b375-301d042a8229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.325216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f189d79-19a8-4f7b-b375-301d042a8229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.326818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f189d79-19a8-4f7b-b375-301d042a8229 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.337594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1225/2416] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:26.341116 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.351207 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff92fb4-cca5-48ee-97c2-6fc8781c8d06 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.355819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.356475 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.356733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.357000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] PATH_INFO: `/v3/projects/57e3c5e98c9c4396a815162c114b0aa2/users/f7370e519994420588e61fdcddea6ccf/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.358914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e3c5e98c9c4396a815162c114b0aa2, user_id=f7370e519994420588e61fdcddea6ccf, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.362084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.372282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1192/2417] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:26 2022] PUT /identity/v3/projects/57e3c5e98c9c4396a815162c114b0aa2/users/f7370e519994420588e61fdcddea6ccf/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:26.383990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d7b5-8172-4b21-80b3-fb2d33251b8d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.384270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d7b5-8172-4b21-80b3-fb2d33251b8d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.384540 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d7b5-8172-4b21-80b3-fb2d33251b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.411672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-aba3d7b5-8172-4b21-80b3-fb2d33251b8d None None] MFA Rules not processed for user `f7370e519994420588e61fdcddea6ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:26.412867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aba3d7b5-8172-4b21-80b3-fb2d33251b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:26.416772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:26.481555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1226/2418] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:26.492955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.494707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:26.567213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41d13-7632-4ce7-a98b-65fad2d327ca admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.571631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7d1c-f522-4b17-b9f1-ecf1bcdcb00d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.572273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7d1c-f522-4b17-b9f1-ecf1bcdcb00d service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.572546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7d1c-f522-4b17-b9f1-ecf1bcdcb00d service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.572812 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7d1c-f522-4b17-b9f1-ecf1bcdcb00d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.582681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7d1c-f522-4b17-b9f1-ecf1bcdcb00d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.584162 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7d1c-f522-4b17-b9f1-ecf1bcdcb00d service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.594077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1193/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:26.622242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3d7b5-8172-4b21-80b3-fb2d33251b8d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.685259 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3d7b5-8172-4b21-80b3-fb2d33251b8d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.690567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.691194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.691475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.691746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] PATH_INFO: `/v3/users/e3cd6f53d1b748008caf3d723172be63` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.692873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3cd6f53d1b748008caf3d723172be63)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.694847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.739772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3cd6f53d1b748008caf3d723172be63'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:26.746652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3cd6f53d1b748008caf3d723172be63'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:26.758790 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3cd6f53d1b748008caf3d723172be63'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:26.759437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3cd6f53d1b748008caf3d723172be63'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:26.766614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3cd6f53d1b748008caf3d723172be63'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:26.772560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1227/2420] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:26 2022] DELETE /identity/v3/users/e3cd6f53d1b748008caf3d723172be63 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:26.777025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e55900-9915-4501-be8f-483fde80a46c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.777350 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e55900-9915-4501-be8f-483fde80a46c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.777629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e55900-9915-4501-be8f-483fde80a46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.797931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-16e55900-9915-4501-be8f-483fde80a46c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:26.799139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-16e55900-9915-4501-be8f-483fde80a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:26.869155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1194/2421] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:26.879774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:26.881427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:26.904536 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:26.948470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb29b4b-0344-4d82-86eb-69b6d1a63469 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:26.952207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:26.952805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:26.953114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:26.953404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:26.962762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:26.964195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:26.974923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1228/2422] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:27.246329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e55900-9915-4501-be8f-483fde80a46c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:27.248089 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-16e55900-9915-4501-be8f-483fde80a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:27.267304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e55900-9915-4501-be8f-483fde80a46c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:27.273001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:27.273639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:27.273918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:27.274301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] PATH_INFO: `/v3/projects/2db8818db6014bf485e0fcc37c6e0e5e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:27.275478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db8818db6014bf485e0fcc37c6e0e5e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:27.277587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:27.299114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:27.324838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db8818db6014bf485e0fcc37c6e0e5e', 'request_id': 'req-1645f961-8ddd-4b70-af2e-a5ab3c38488e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.331513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db8818db6014bf485e0fcc37c6e0e5e', 'request_id': 'req-1645f961-8ddd-4b70-af2e-a5ab3c38488e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.337106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db8818db6014bf485e0fcc37c6e0e5e', 'request_id': 'req-1645f961-8ddd-4b70-af2e-a5ab3c38488e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.352005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] The token cache is being invalidate because project 2db8818db6014bf485e0fcc37c6e0e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:27.352368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.353128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.354639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1195/2423] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:27 2022] DELETE /identity/v3/projects/2db8818db6014bf485e0fcc37c6e0e5e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:27.358918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:27.368464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62192b1-26ab-4a6c-a159-59b865ef4cff service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:27.423277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:27.423891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:27.424170 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:27.424443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] PATH_INFO: `/v3/users/f7370e519994420588e61fdcddea6ccf` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:27.425583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7370e519994420588e61fdcddea6ccf)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:27.427517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:27.473698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7370e519994420588e61fdcddea6ccf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.480736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7370e519994420588e61fdcddea6ccf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.492488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7370e519994420588e61fdcddea6ccf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.493151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7370e519994420588e61fdcddea6ccf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.501402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-50c91a7b-ac52-4097-9a6d-7fbc698851b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7370e519994420588e61fdcddea6ccf'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:27.507279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1229/2424] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:27 2022] DELETE /identity/v3/users/f7370e519994420588e61fdcddea6ccf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:28.195931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.197358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:28.212240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645f961-8ddd-4b70-af2e-a5ab3c38488e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:28.286133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:28.286957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.287301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.287659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] PATH_INFO: `/v3/projects/57e3c5e98c9c4396a815162c114b0aa2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.289236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=57e3c5e98c9c4396a815162c114b0aa2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:28.291429 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:28.346227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57e3c5e98c9c4396a815162c114b0aa2', 'request_id': 'req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:28.352884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57e3c5e98c9c4396a815162c114b0aa2', 'request_id': 'req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:28.358778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57e3c5e98c9c4396a815162c114b0aa2', 'request_id': 'req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:28.372066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] The token cache is being invalidate because project 57e3c5e98c9c4396a815162c114b0aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:28.372472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:28.373128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5c81cc36-1a7e-4378-8e6f-193645e1bd1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:28.374688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1196/2425] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:28 2022] DELETE /identity/v3/projects/57e3c5e98c9c4396a815162c114b0aa2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:28.380470 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb124-24f1-4366-af27-c171ae568ceb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.381023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb124-24f1-4366-af27-c171ae568ceb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.381141 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb124-24f1-4366-af27-c171ae568ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.402237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cf1eb124-24f1-4366-af27-c171ae568ceb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:28.403498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cf1eb124-24f1-4366-af27-c171ae568ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:28.459058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:28.482447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1230/2426] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:28.490585 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb677139-b624-4283-83a3-bfe57ed804b0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.490869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb677139-b624-4283-83a3-bfe57ed804b0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.491140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb677139-b624-4283-83a3-bfe57ed804b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.511211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cb677139-b624-4283-83a3-bfe57ed804b0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:28.512438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cb677139-b624-4283-83a3-bfe57ed804b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:28.524171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1197/2427] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:28.528380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eb124-24f1-4366-af27-c171ae568ceb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.538943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eb124-24f1-4366-af27-c171ae568ceb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:28.543698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:28.544293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.544573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.544849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.545526 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:28.545829 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:28.547989 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:28.559554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1231/2428] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:28.563974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb677139-b624-4283-83a3-bfe57ed804b0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.573065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb677139-b624-4283-83a3-bfe57ed804b0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:28.577733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:28.578321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.578600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.578869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.579490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:28.581423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:28.611650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1198/2429] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:28 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:28.615731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.625990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e929a6-ac7e-414c-ae48-7371cc0b698f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:28.688125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:28.688732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.689008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.689302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.689924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:28.691840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:28.714171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1232/2430] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:28 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:28.718363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.729013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4531a0-03d6-49a5-81e6-17f8eac53f23 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:28.733641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:28.734192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.734459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.734729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.735355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:28.736918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:28.738906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:28.748456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1199/2431] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:28.752594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.762351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b46677-e604-4a3c-b6c4-c9f7a46295d1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:28.767431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e5847-6e9f-4a4a-adb0-05e667ed0301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:28.768011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5847-6e9f-4a4a-adb0-05e667ed0301 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.768283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5847-6e9f-4a4a-adb0-05e667ed0301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.768551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5847-6e9f-4a4a-adb0-05e667ed0301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.769175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e5847-6e9f-4a4a-adb0-05e667ed0301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:28.770774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e5847-6e9f-4a4a-adb0-05e667ed0301 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:28.780651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1233/2432] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:28.785329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.795206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98d9c8-e5b7-4626-9a60-dc0901b5cdef admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:28.799823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:28.800459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.800738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.801011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] PATH_INFO: `/v3/projects/a6b57c64ee794caca3612738b474acbe/users/a328abb4a365467faa5aa5064144da8e/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.815077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b57c64ee794caca3612738b474acbe, user_id=a328abb4a365467faa5aa5064144da8e, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:28.817760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:28.830742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1200/2433] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:28 2022] PUT /identity/v3/projects/a6b57c64ee794caca3612738b474acbe/users/a328abb4a365467faa5aa5064144da8e/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:28.840870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f904175-e8f0-43c3-8d88-c7a293857a3c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:28.841154 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f904175-e8f0-43c3-8d88-c7a293857a3c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:28.841451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f904175-e8f0-43c3-8d88-c7a293857a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:28.868910 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1f904175-e8f0-43c3-8d88-c7a293857a3c None None] MFA Rules not processed for user `a328abb4a365467faa5aa5064144da8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:28.870152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1f904175-e8f0-43c3-8d88-c7a293857a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:28.931179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1234/2434] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:28.935524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:28.996454 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787df03-4831-49bd-860c-5b2ef70e2582 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.006781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.007408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.007689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.007965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.008617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.011082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.042403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1201/2435] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:28 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:29.043605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f904175-e8f0-43c3-8d88-c7a293857a3c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:29.056120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f904175-e8f0-43c3-8d88-c7a293857a3c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.091484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:29.116371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.116973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.117285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.117624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.118306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.120280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.143719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1235/2436] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:29 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:29.147634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:29.158910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56843380-993e-41d2-befd-21f398001e0d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.164295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.165325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.165624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.165893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.166549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.168153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.183524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1202/2437] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:29.184610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:29.195103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855357a-c97c-470f-9ff0-96d9654c7b5c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.199842 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.200620 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.200902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.201169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] PATH_INFO: `/v3/projects/cee7f3dfad93423995681997cf617500/users/45c359206c474f15b1c601411231b593/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.217115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee7f3dfad93423995681997cf617500, user_id=45c359206c474f15b1c601411231b593, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.219789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.232898 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1236/2438] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:29 2022] PUT /identity/v3/projects/cee7f3dfad93423995681997cf617500/users/45c359206c474f15b1c601411231b593/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:29.236904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:29.247287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76faf92-8a0f-4a7e-9448-625898956e8f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.305281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d6d7d-d9d7-49df-8c12-c03af74a17af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.305847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d6d7d-d9d7-49df-8c12-c03af74a17af admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.306235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d6d7d-d9d7-49df-8c12-c03af74a17af admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.306397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d6d7d-d9d7-49df-8c12-c03af74a17af admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.307025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d6d7d-d9d7-49df-8c12-c03af74a17af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.308618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d6d7d-d9d7-49df-8c12-c03af74a17af admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.319239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1203/2439] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:29.323406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:29.334117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421073e-b686-4d75-9282-63b231c33ed4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.338682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.339324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.339603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.339876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] PATH_INFO: `/v3/projects/cee7f3dfad93423995681997cf617500/users/45c359206c474f15b1c601411231b593/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.341806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee7f3dfad93423995681997cf617500, user_id=45c359206c474f15b1c601411231b593, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.344452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.355612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1237/2440] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:29 2022] PUT /identity/v3/projects/cee7f3dfad93423995681997cf617500/users/45c359206c474f15b1c601411231b593/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:29.369707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c39b3-647f-4408-92de-5c9c24c889d4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.369996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c39b3-647f-4408-92de-5c9c24c889d4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.370291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c39b3-647f-4408-92de-5c9c24c889d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.401191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-762c39b3-647f-4408-92de-5c9c24c889d4 None None] MFA Rules not processed for user `45c359206c474f15b1c601411231b593`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:29.402426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-762c39b3-647f-4408-92de-5c9c24c889d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:29.436548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:29.485357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1204/2441] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:29.494787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:29.497479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:29.561899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87fa18-cb9b-46c6-bd00-ec31751c87bb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.566361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef38fc0-b90c-4e51-bbf4-a710783b8ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.566968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef38fc0-b90c-4e51-bbf4-a710783b8ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.567270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef38fc0-b90c-4e51-bbf4-a710783b8ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.567506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef38fc0-b90c-4e51-bbf4-a710783b8ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.577996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef38fc0-b90c-4e51-bbf4-a710783b8ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.579510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef38fc0-b90c-4e51-bbf4-a710783b8ec3 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.590826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1238/2442] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:29.700792 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c39b3-647f-4408-92de-5c9c24c889d4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:29.710435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c39b3-647f-4408-92de-5c9c24c889d4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:29.714085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:29.714663 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:29.714940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:29.715214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:29.773880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:29.775900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:29.787874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1205/2443] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:31.245384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efbafd-fb13-410b-bcb1-d3750eff819e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:31.245384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efbafd-fb13-410b-bcb1-d3750eff819e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:31.245384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efbafd-fb13-410b-bcb1-d3750eff819e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:31.264896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-71efbafd-fb13-410b-bcb1-d3750eff819e None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:31.266136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-71efbafd-fb13-410b-bcb1-d3750eff819e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:31.324295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1239/2444] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:31.336590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:31.338077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:31.416171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e132b-09b1-4b6b-bb68-85a4118f55ef service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:31.421117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a164927-3025-499c-b721-2ceff1f46d9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:31.422369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a164927-3025-499c-b721-2ceff1f46d9a service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:31.422648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a164927-3025-499c-b721-2ceff1f46d9a service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:31.422914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a164927-3025-499c-b721-2ceff1f46d9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:31.430800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a164927-3025-499c-b721-2ceff1f46d9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:31.432312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a164927-3025-499c-b721-2ceff1f46d9a service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:31.440259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1206/2445] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:03:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:03:32.444681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efbafd-fb13-410b-bcb1-d3750eff819e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:32.483234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:32.507331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efbafd-fb13-410b-bcb1-d3750eff819e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:32.512001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:32.512644 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:32.512919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:32.513184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] PATH_INFO: `/v3/users/a328abb4a365467faa5aa5064144da8e` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:32.514335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a328abb4a365467faa5aa5064144da8e)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:32.516306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:32.565854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a328abb4a365467faa5aa5064144da8e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:32.572855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a328abb4a365467faa5aa5064144da8e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:32.584583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a328abb4a365467faa5aa5064144da8e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:32.585286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a328abb4a365467faa5aa5064144da8e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:32.592067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a328abb4a365467faa5aa5064144da8e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:32.598023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1240/2446] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:32 2022] DELETE /identity/v3/users/a328abb4a365467faa5aa5064144da8e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:32.602448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:32.602914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:32.603012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:32.623032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:32.624230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:32.673673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:32.682943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1207/2447] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:32.692692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:32.694103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:32.749543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac15b9-3d2f-4fb3-ab11-ecae2cb9cd13 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:32.753615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:32.754205 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:32.754482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:32.754752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:32.763386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:32.764811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:32.774611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1241/2448] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:33.037600 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:33.039213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:33.052953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f7bb6-7add-46dc-8fce-d57d01fa61d6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:33.059322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:33.060383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:33.060585 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:33.060874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] PATH_INFO: `/v3/projects/a6b57c64ee794caca3612738b474acbe` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:33.062054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b57c64ee794caca3612738b474acbe)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:33.064574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:33.113277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b57c64ee794caca3612738b474acbe', 'request_id': 'req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.119711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b57c64ee794caca3612738b474acbe', 'request_id': 'req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.126177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b57c64ee794caca3612738b474acbe', 'request_id': 'req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.142788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] The token cache is being invalidate because project a6b57c64ee794caca3612738b474acbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:33.143206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.144487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.146823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1208/2449] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:33 2022] DELETE /identity/v3/projects/a6b57c64ee794caca3612738b474acbe => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:33.151718 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:33.162088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfadc9-e714-4b49-91f6-a5948473bb2d service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:33.222481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:33.223115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:33.223424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:33.223701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] PATH_INFO: `/v3/users/45c359206c474f15b1c601411231b593` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:33.224858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c359206c474f15b1c601411231b593)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:33.227022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:33.228600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:33.276883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c359206c474f15b1c601411231b593'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.283946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c359206c474f15b1c601411231b593'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.295922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c359206c474f15b1c601411231b593'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.296619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c359206c474f15b1c601411231b593'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.304503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae7383a6-0b43-4d01-8bf1-06f46d82478d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c359206c474f15b1c601411231b593'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.310481 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1242/2450] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:33 2022] DELETE /identity/v3/users/45c359206c474f15b1c601411231b593 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:33.593005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:33.596470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:33.610118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9bb41-610b-45f6-ad1e-090ada9b5bbc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:33.670258 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:33.670887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:33.671166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:33.671439 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] PATH_INFO: `/v3/projects/cee7f3dfad93423995681997cf617500` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:33.673218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee7f3dfad93423995681997cf617500)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:33.675108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:33.719681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee7f3dfad93423995681997cf617500', 'request_id': 'req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.725753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee7f3dfad93423995681997cf617500', 'request_id': 'req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.731176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee7f3dfad93423995681997cf617500', 'request_id': 'req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.744424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] The token cache is being invalidate because project cee7f3dfad93423995681997cf617500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:33.744709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.745355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-889d1d7f-d170-4e37-aa61-ceb2b5b9e49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:33.746840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1209/2451] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:33 2022] DELETE /identity/v3/projects/cee7f3dfad93423995681997cf617500 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:33.752138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f29a1-62e8-486c-8418-f371ac397ee6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:33.752591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f29a1-62e8-486c-8418-f371ac397ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:33.752686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f29a1-62e8-486c-8418-f371ac397ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:33.771694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-820f29a1-62e8-486c-8418-f371ac397ee6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:33.772875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-820f29a1-62e8-486c-8418-f371ac397ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:33.829855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1243/2452] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:33.837339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5fd7d-ad63-46ac-aacb-1226e9c18b35 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:33.837630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5fd7d-ad63-46ac-aacb-1226e9c18b35 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:33.837905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5fd7d-ad63-46ac-aacb-1226e9c18b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:33.857856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c9f5fd7d-ad63-46ac-aacb-1226e9c18b35 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:33.859062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c9f5fd7d-ad63-46ac-aacb-1226e9c18b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:33.866883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1210/2453] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:33.871093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f29a1-62e8-486c-8418-f371ac397ee6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:33.880588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f29a1-62e8-486c-8418-f371ac397ee6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:33.885185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:33.885796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:33.886075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:33.886346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:33.886981 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:33.887283 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:33.888902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:33.900594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1244/2454] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:33.904619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5fd7d-ad63-46ac-aacb-1226e9c18b35 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:33.911040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:33.915066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5fd7d-ad63-46ac-aacb-1226e9c18b35 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:33.921103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:33.921907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:33.922408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:33.922743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:33.923565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:33.925481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:33.951414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1211/2455] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:33 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:33.955920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:33.965838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b477f-8ca2-4037-8f27-eb68a37ccded admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.021256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.021855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.022131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.022399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.023032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.024943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.037344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:34.048381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1245/2456] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:33 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:34.052632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.063057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803379a4-b4e3-47f4-adab-67013cba5233 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.067658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.068244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.068518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.068785 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.069444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.071015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.081155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1212/2457] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:34.085007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.095747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8618d20-64a9-438e-a001-ea8d1ddb4b37 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.100456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e7d72-4bbb-4253-aaea-14195c6a6f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.101043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e7d72-4bbb-4253-aaea-14195c6a6f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.101359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e7d72-4bbb-4253-aaea-14195c6a6f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.101717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e7d72-4bbb-4253-aaea-14195c6a6f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.102350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e7d72-4bbb-4253-aaea-14195c6a6f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.103933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e7d72-4bbb-4253-aaea-14195c6a6f8e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.114295 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1246/2458] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:34.118492 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.128157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062c41a-90b5-4ec7-ad5f-379fd6b14794 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.132689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.133348 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.133624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.133893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] PATH_INFO: `/v3/projects/17ec98308c874ed1b235fce175f93cd0/users/6569c6c6eb4e4bc0b2b8b6461e83bc28/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.148221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ec98308c874ed1b235fce175f93cd0, user_id=6569c6c6eb4e4bc0b2b8b6461e83bc28, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.150868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.164504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1213/2459] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:34 2022] PUT /identity/v3/projects/17ec98308c874ed1b235fce175f93cd0/users/6569c6c6eb4e4bc0b2b8b6461e83bc28/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:34.174696 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cf6ec-b02f-4495-8636-85d94d7fd510 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.174977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cf6ec-b02f-4495-8636-85d94d7fd510 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.175251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cf6ec-b02f-4495-8636-85d94d7fd510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.202175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8c3cf6ec-b02f-4495-8636-85d94d7fd510 None None] MFA Rules not processed for user `6569c6c6eb4e4bc0b2b8b6461e83bc28`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:34.203925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8c3cf6ec-b02f-4495-8636-85d94d7fd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:34.264891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1247/2460] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:34.270121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.330959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9494b0-9226-4376-852b-43c6096dd868 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.335830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.336361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.336637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.336902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.337572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.339452 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.365607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1214/2461] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:34 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:34.369783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cf6ec-b02f-4495-8636-85d94d7fd510 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.381863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cf6ec-b02f-4495-8636-85d94d7fd510 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.435591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.436192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.436471 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.436741 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.437881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.439791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.461659 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1248/2462] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:34 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:34.465611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.475815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff80bbc-aec2-41ff-8339-803d2635b048 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.480395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.480981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.481328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.481601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.482260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.483823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.494170 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1215/2463] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:03:34.498049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.508332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a1b3-fa18-4260-959f-907591614c21 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.513048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.513707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.513983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.514253 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] PATH_INFO: `/v3/projects/21e68e9b46f64072a3f2326b1f9e8497/users/430822da4ca846edbbb8902284de7ac4/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.529116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e68e9b46f64072a3f2326b1f9e8497, user_id=430822da4ca846edbbb8902284de7ac4, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.532102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.545265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1249/2464] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:34 2022] PUT /identity/v3/projects/21e68e9b46f64072a3f2326b1f9e8497/users/430822da4ca846edbbb8902284de7ac4/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:34.549081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.559165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea0ae-720c-4aab-b984-08d6cab6e92a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.616443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b4f32-a28c-4cd9-9cc4-5b9d7d8c2550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.617066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b4f32-a28c-4cd9-9cc4-5b9d7d8c2550 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.617371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b4f32-a28c-4cd9-9cc4-5b9d7d8c2550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.617540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b4f32-a28c-4cd9-9cc4-5b9d7d8c2550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.618148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b4f32-a28c-4cd9-9cc4-5b9d7d8c2550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.619740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b4f32-a28c-4cd9-9cc4-5b9d7d8c2550 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.633469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1216/2465] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:34.634658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.645568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2226-18aa-46d9-b57c-5002b03bf5b7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.650133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.650770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.651044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.651305 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] PATH_INFO: `/v3/projects/21e68e9b46f64072a3f2326b1f9e8497/users/430822da4ca846edbbb8902284de7ac4/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.653232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e68e9b46f64072a3f2326b1f9e8497, user_id=430822da4ca846edbbb8902284de7ac4, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.656122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.666062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1250/2466] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:34 2022] PUT /identity/v3/projects/21e68e9b46f64072a3f2326b1f9e8497/users/430822da4ca846edbbb8902284de7ac4/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:34.677513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167851-cf5e-4f54-a67f-43a3aa446d8a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.677801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167851-cf5e-4f54-a67f-43a3aa446d8a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.678077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167851-cf5e-4f54-a67f-43a3aa446d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.705771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-dd167851-cf5e-4f54-a67f-43a3aa446d8a None None] MFA Rules not processed for user `430822da4ca846edbbb8902284de7ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:34.706963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-dd167851-cf5e-4f54-a67f-43a3aa446d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:34.711849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:34.774658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1217/2467] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:34 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:34.785501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:34.787215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:34.818395 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:34.846479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cca26-f77e-4328-accf-254e50151647 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:34.850159 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa6229-b618-4a88-9d2a-5bb0e02ddba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:34.850744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa6229-b618-4a88-9d2a-5bb0e02ddba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:34.851017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa6229-b618-4a88-9d2a-5bb0e02ddba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:34.851281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa6229-b618-4a88-9d2a-5bb0e02ddba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:34.860532 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa6229-b618-4a88-9d2a-5bb0e02ddba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:34.861975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa6229-b618-4a88-9d2a-5bb0e02ddba4 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:34.871160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1251/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:35.003382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd167851-cf5e-4f54-a67f-43a3aa446d8a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:35.065080 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd167851-cf5e-4f54-a67f-43a3aa446d8a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:35.069723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:35.070344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:35.070620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:35.070891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] PATH_INFO: `/v3/users/6569c6c6eb4e4bc0b2b8b6461e83bc28` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:35.072029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6569c6c6eb4e4bc0b2b8b6461e83bc28)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:35.073967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:35.119736 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6569c6c6eb4e4bc0b2b8b6461e83bc28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.131176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6569c6c6eb4e4bc0b2b8b6461e83bc28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.137882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6569c6c6eb4e4bc0b2b8b6461e83bc28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.144945 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6569c6c6eb4e4bc0b2b8b6461e83bc28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.145573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6569c6c6eb4e4bc0b2b8b6461e83bc28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.151441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1218/2469] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:35 2022] DELETE /identity/v3/users/6569c6c6eb4e4bc0b2b8b6461e83bc28 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:35.156650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:35.156939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:35.157219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:35.177645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:35.178627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:35.237424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1252/2470] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:35.247101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:35.248505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:35.274631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:35.308833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e7648-b91a-4780-9949-ebda229c975f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:35.312667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:35.313296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:35.313572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:35.313839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:35.322717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:35.324124 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:35.333740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1219/2471] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:35.602835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:35.604243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:35.615938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854fe79-a8e2-4df9-a8c6-50ff009d0fac None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:35.620439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:35.621062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:35.621373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:35.621641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] PATH_INFO: `/v3/projects/17ec98308c874ed1b235fce175f93cd0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:35.622750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ec98308c874ed1b235fce175f93cd0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:35.624647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:35.674352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ec98308c874ed1b235fce175f93cd0', 'request_id': 'req-e9812c50-182b-4b46-ba33-728ae64e2ec8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.680812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ec98308c874ed1b235fce175f93cd0', 'request_id': 'req-e9812c50-182b-4b46-ba33-728ae64e2ec8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.686294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ec98308c874ed1b235fce175f93cd0', 'request_id': 'req-e9812c50-182b-4b46-ba33-728ae64e2ec8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.698498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] The token cache is being invalidate because project 17ec98308c874ed1b235fce175f93cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:35.698810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.699446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.700909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1253/2472] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:35 2022] DELETE /identity/v3/projects/17ec98308c874ed1b235fce175f93cd0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:35.705363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:35.714731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813c39c-d763-4f17-96cc-40751833eb56 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:35.770108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:35.770745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:35.771027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:35.771302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] PATH_INFO: `/v3/users/430822da4ca846edbbb8902284de7ac4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:35.772425 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=430822da4ca846edbbb8902284de7ac4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:35.774375 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:35.820546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430822da4ca846edbbb8902284de7ac4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.836848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430822da4ca846edbbb8902284de7ac4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.845433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430822da4ca846edbbb8902284de7ac4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.853400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430822da4ca846edbbb8902284de7ac4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.854062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d0669b1d-d7b6-4737-80e7-c98f42a7b586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430822da4ca846edbbb8902284de7ac4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:35.860070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1220/2473] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:35 2022] DELETE /identity/v3/users/430822da4ca846edbbb8902284de7ac4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:36.131155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.132635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:36.145695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9812c50-182b-4b46-ba33-728ae64e2ec8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.200132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:36.211610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.212575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.212924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.213214 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] PATH_INFO: `/v3/projects/21e68e9b46f64072a3f2326b1f9e8497` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.215133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e68e9b46f64072a3f2326b1f9e8497)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.217046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.263685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e68e9b46f64072a3f2326b1f9e8497', 'request_id': 'req-374e6fce-3fff-464b-a92e-387efe0b5354'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:36.269800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e68e9b46f64072a3f2326b1f9e8497', 'request_id': 'req-374e6fce-3fff-464b-a92e-387efe0b5354'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:36.275278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e68e9b46f64072a3f2326b1f9e8497', 'request_id': 'req-374e6fce-3fff-464b-a92e-387efe0b5354'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:36.287371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] The token cache is being invalidate because project 21e68e9b46f64072a3f2326b1f9e8497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:36.287658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:36.288270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-374e6fce-3fff-464b-a92e-387efe0b5354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:36.289755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1254/2474] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:36 2022] DELETE /identity/v3/projects/21e68e9b46f64072a3f2326b1f9e8497 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:36.294611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be445a9-29b3-4cac-bffb-33792f635922 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.294897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be445a9-29b3-4cac-bffb-33792f635922 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.295165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be445a9-29b3-4cac-bffb-33792f635922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.315850 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0be445a9-29b3-4cac-bffb-33792f635922 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:36.317062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0be445a9-29b3-4cac-bffb-33792f635922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:36.356016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:36.377853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1221/2475] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:36.385626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec9a0a-ea34-4e5d-909a-183b238a9fbe None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.385915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec9a0a-ea34-4e5d-909a-183b238a9fbe None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.386193 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec9a0a-ea34-4e5d-909a-183b238a9fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.406884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-01ec9a0a-ea34-4e5d-909a-183b238a9fbe None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:36.408089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-01ec9a0a-ea34-4e5d-909a-183b238a9fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:36.416067 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1255/2476] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:36.420281 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be445a9-29b3-4cac-bffb-33792f635922 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.430306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be445a9-29b3-4cac-bffb-33792f635922 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.434896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.435484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.435764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.436034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.436672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:36.436974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.439207 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.450372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1222/2477] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:36.454690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec9a0a-ea34-4e5d-909a-183b238a9fbe None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.463826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec9a0a-ea34-4e5d-909a-183b238a9fbe None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.469848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.470431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.470755 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.471022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.471640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.473610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.501570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1256/2478] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:36 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:36.507337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.521560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00004d3-fc13-4a04-b811-149088d8e747 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.578420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.579017 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.579295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.579562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.580180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.582123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.604490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1223/2479] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:36 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:36.608613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.619327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc0104-a9a2-44e9-859f-bb7f04cf9771 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.623918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.624502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.624779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.625078 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.625766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.627343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.636937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1257/2480] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:36.640601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.650642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba1186-cfb7-4dde-bf13-c2bd202ae54a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.656806 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdb19d-b61c-4720-b112-f0fcf5d86fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.657529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdb19d-b61c-4720-b112-f0fcf5d86fbc admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.657823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdb19d-b61c-4720-b112-f0fcf5d86fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.658082 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdb19d-b61c-4720-b112-f0fcf5d86fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.658712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdb19d-b61c-4720-b112-f0fcf5d86fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.660357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdb19d-b61c-4720-b112-f0fcf5d86fbc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.671829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1224/2481] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:03:36.676750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.687169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eceaa-cc84-415a-964c-053c40d6e9f2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.691869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.692512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.692792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.693069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] PATH_INFO: `/v3/projects/f970622440bb427382b99e4a68d8746e/users/ccd1440631a94f7b87403dad65292579/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.708650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f970622440bb427382b99e4a68d8746e, user_id=ccd1440631a94f7b87403dad65292579, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.711535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.726033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1258/2482] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:36 2022] PUT /identity/v3/projects/f970622440bb427382b99e4a68d8746e/users/ccd1440631a94f7b87403dad65292579/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:36.731045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe482a4a-0517-4de5-a9b6-bd9f23214ad6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.731500 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe482a4a-0517-4de5-a9b6-bd9f23214ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.731645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe482a4a-0517-4de5-a9b6-bd9f23214ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.751345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-fe482a4a-0517-4de5-a9b6-bd9f23214ad6 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:36.752548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fe482a4a-0517-4de5-a9b6-bd9f23214ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:36.817027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1225/2483] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:36.826839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:36.829115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:36.892638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8c2ea-2348-4bd8-9e57-1121b8c0b30f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:36.896336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:36.896977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:36.897284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:36.897557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:36.907777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:36.909251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:36.918509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1259/2484] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:37.586984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af228-549e-4bd3-bd53-49d5cc976544 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:37.587379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af228-549e-4bd3-bd53-49d5cc976544 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:37.587758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af228-549e-4bd3-bd53-49d5cc976544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:37.619141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a43af228-549e-4bd3-bd53-49d5cc976544 None None] MFA Rules not processed for user `ccd1440631a94f7b87403dad65292579`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:37.620388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a43af228-549e-4bd3-bd53-49d5cc976544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:37.663248 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:37.691709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1226/2485] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:37.696613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:37.706573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301a505-8484-45b1-9e90-65d215b69f0e service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:37.711303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:37.711910 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:37.712186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:37.712454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:37.713096 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:37.715064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:37.738805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:37.750705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1260/2486] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:37 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:37.752067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43af228-549e-4bd3-bd53-49d5cc976544 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:37.763234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43af228-549e-4bd3-bd53-49d5cc976544 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:37.822409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:37.823106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:37.823394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:37.823669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:37.824315 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:37.826274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:37.851123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1227/2487] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:37 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:37.852294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:37.864341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9584d26-18e5-4c14-97bd-0a4063516121 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:37.869784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:37.870432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:37.870804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:37.871086 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:37.871739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:37.873397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:37.888211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1261/2488] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:37.892694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:37.903816 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8e63-08d3-4862-ac32-509239804430 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:37.908815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:37.909510 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:37.909791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:37.910063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] PATH_INFO: `/v3/projects/9b3f3b15aa404ddfb641fe254765dcb8/users/daf69c0483224a3bb2b9aefe96e6d458/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:37.926303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3f3b15aa404ddfb641fe254765dcb8, user_id=daf69c0483224a3bb2b9aefe96e6d458, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:37.928987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:37.943302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1228/2489] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:37 2022] PUT /identity/v3/projects/9b3f3b15aa404ddfb641fe254765dcb8/users/daf69c0483224a3bb2b9aefe96e6d458/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:37.947246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:37.957229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02bad-0e41-4577-ab6e-74a373ba84da admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:38.012761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc772856-efca-4646-8add-083f131b13df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:38.013317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc772856-efca-4646-8add-083f131b13df admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:38.013601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc772856-efca-4646-8add-083f131b13df admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:38.013877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc772856-efca-4646-8add-083f131b13df admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:38.014518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc772856-efca-4646-8add-083f131b13df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:38.016177 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc772856-efca-4646-8add-083f131b13df admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:38.026426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1262/2490] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:38.030862 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:38.041715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c4f42-0cae-4c22-a818-abcba0f210dd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:38.046188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:38.047621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:38.047893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:38.048157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] PATH_INFO: `/v3/projects/9b3f3b15aa404ddfb641fe254765dcb8/users/daf69c0483224a3bb2b9aefe96e6d458/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:38.050417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3f3b15aa404ddfb641fe254765dcb8, user_id=daf69c0483224a3bb2b9aefe96e6d458, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:38.053065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:38.063825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1229/2491] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:38 2022] PUT /identity/v3/projects/9b3f3b15aa404ddfb641fe254765dcb8/users/daf69c0483224a3bb2b9aefe96e6d458/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:38.708533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:38.709014 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:38.709112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:38.737052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] MFA Rules not processed for user `daf69c0483224a3bb2b9aefe96e6d458`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:38.738296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:38.810208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1263/2492] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:38.821975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:38.823677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:38.880725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c462cc-f8b5-435c-b01f-30912b82e0e9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:38.884283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98d681-cb56-4af6-88f6-05174572045b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:38.884885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98d681-cb56-4af6-88f6-05174572045b service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:38.885158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98d681-cb56-4af6-88f6-05174572045b service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:38.885446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98d681-cb56-4af6-88f6-05174572045b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:38.929471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:38.942899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98d681-cb56-4af6-88f6-05174572045b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:38.944310 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98d681-cb56-4af6-88f6-05174572045b service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:38.953636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1230/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:39.763324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:39.764838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:39.828720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16f305-f1d4-4c5f-950f-f69799d28ac7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:39.833171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:39.833817 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:39.834091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:39.834370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] PATH_INFO: `/v3/users/8949b50e8cb6424182aa97c8a171bd5c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:39.835461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8949b50e8cb6424182aa97c8a171bd5c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:39.837413 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:39.849921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:39.884825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8949b50e8cb6424182aa97c8a171bd5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:39.892163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8949b50e8cb6424182aa97c8a171bd5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:39.903952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8949b50e8cb6424182aa97c8a171bd5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:39.904613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8949b50e8cb6424182aa97c8a171bd5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:39.912625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8949b50e8cb6424182aa97c8a171bd5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:39.918432 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1264/2494] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:39 2022] DELETE /identity/v3/users/8949b50e8cb6424182aa97c8a171bd5c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:39.922926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:39.923395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:39.923575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:39.942018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:39.943189 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:40.000223 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1231/2495] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:40.009960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:40.011402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:40.067116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc57ae1-67fc-4af9-a3df-904df78e5e6f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:40.070681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:40.071274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:40.071550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:40.071819 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:40.087065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:40.088491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:40.098642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1265/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:40.288146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:40.290312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:40.331487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:40.333071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:40.396368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c45664-c0aa-4b5c-b3a3-4098c9542e66 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:40.401453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1adf0-1d6e-4f52-ae5e-dbbc5873fa0a service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:40.404443 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:40.405165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:40.405915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:40.406361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:40.406647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:40.406730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:40.407034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:40.407389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:40.408531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:40.409261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:40.463628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:40.474253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:40.475861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:40.483690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:40.485171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:40.485402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1232/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:40.486240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:40.487564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:40.501742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1266/2498] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:03:40.502105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9ceab-dcc5-4f5a-be7d-6a2b6625b798 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:40.507141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:40.507769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:40.508054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:40.508312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] PATH_INFO: `/v3/projects/adfe6967086941fb84a3f161b5e4d22c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:40.509469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adfe6967086941fb84a3f161b5e4d22c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:40.511514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:40.537606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:40.569368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adfe6967086941fb84a3f161b5e4d22c', 'request_id': 'req-9d0daab3-5a81-4b27-90e0-f380f0047ec5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.576596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adfe6967086941fb84a3f161b5e4d22c', 'request_id': 'req-9d0daab3-5a81-4b27-90e0-f380f0047ec5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.584024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adfe6967086941fb84a3f161b5e4d22c', 'request_id': 'req-9d0daab3-5a81-4b27-90e0-f380f0047ec5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.601445 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] The token cache is being invalidate because project adfe6967086941fb84a3f161b5e4d22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:40.601833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.603157 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.605262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1233/2499] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:40 2022] DELETE /identity/v3/projects/adfe6967086941fb84a3f161b5e4d22c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:40.609010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:40.621173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e871b1-5698-4320-bf72-2dff060507da service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:40.686219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:40.686852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:40.687260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:40.687646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] PATH_INFO: `/v3/users/6f85b410142a4efc99615aeb88d00527` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:40.689564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f85b410142a4efc99615aeb88d00527)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:40.691812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:40.750507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f85b410142a4efc99615aeb88d00527'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.773167 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f85b410142a4efc99615aeb88d00527'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.787613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f85b410142a4efc99615aeb88d00527'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.788346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f85b410142a4efc99615aeb88d00527'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.797498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-09e05a00-d080-4df0-ac02-d32171f8a51a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f85b410142a4efc99615aeb88d00527'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:40.803912 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1267/2500] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:40 2022] DELETE /identity/v3/users/6f85b410142a4efc99615aeb88d00527 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:41.078534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:41.080123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:41.098357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0daab3-5a81-4b27-90e0-f380f0047ec5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:41.157924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:41.158591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.158898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.159168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] PATH_INFO: `/v3/projects/1f17737418624a9bacbc498a8e3e8b9a` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.160311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f17737418624a9bacbc498a8e3e8b9a)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:41.162313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:41.206943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f17737418624a9bacbc498a8e3e8b9a', 'request_id': 'req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:41.212903 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f17737418624a9bacbc498a8e3e8b9a', 'request_id': 'req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:41.218379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f17737418624a9bacbc498a8e3e8b9a', 'request_id': 'req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:41.230894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] The token cache is being invalidate because project 1f17737418624a9bacbc498a8e3e8b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:41.231177 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:41.231790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3e6ecb4a-aba7-49d0-8a5f-13ba8b314dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:41.233257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1234/2501] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:41 2022] DELETE /identity/v3/projects/1f17737418624a9bacbc498a8e3e8b9a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:41.238647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6ff4-f77a-4bbe-b3e8-a7e28cc7e732 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.238930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6ff4-f77a-4bbe-b3e8-a7e28cc7e732 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.239201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6ff4-f77a-4bbe-b3e8-a7e28cc7e732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.258202 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c1bf6ff4-f77a-4bbe-b3e8-a7e28cc7e732 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:41.259326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c1bf6ff4-f77a-4bbe-b3e8-a7e28cc7e732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:41.317463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1268/2502] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:41.326405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43d43-aa55-49d0-8c4f-ebc67b6f01d7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.326689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43d43-aa55-49d0-8c4f-ebc67b6f01d7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.326963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43d43-aa55-49d0-8c4f-ebc67b6f01d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.347517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-56a43d43-aa55-49d0-8c4f-ebc67b6f01d7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:41.348712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-56a43d43-aa55-49d0-8c4f-ebc67b6f01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:41.356832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1235/2503] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:41.361250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf6ff4-f77a-4bbe-b3e8-a7e28cc7e732 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:41.365995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:41.371590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf6ff4-f77a-4bbe-b3e8-a7e28cc7e732 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:41.377257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:41.378021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.378301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.378577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.379217 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:41.379518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:41.381131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:41.392894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1269/2504] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:41.397146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43d43-aa55-49d0-8c4f-ebc67b6f01d7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:41.409474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43d43-aa55-49d0-8c4f-ebc67b6f01d7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:41.416385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:41.416994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.417377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.417655 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.418299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:41.420352 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:41.447004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1236/2505] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:41 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:41.451419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:41.461791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520c14-2a5a-4fb7-a094-eb41f04f7fde admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:41.516825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:41.517371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.517649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.517921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.518543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:41.520460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:41.541895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1270/2506] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:41 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:41.545924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:41.556269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778554f-ff38-456a-adc6-b265c316001e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:41.560912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:41.561530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.561813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.562086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.562742 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:41.564328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:41.566342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:41.576996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1237/2507] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:41.581570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:41.593169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61051907-9304-443f-9138-e9ac610fe270 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:41.598787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe4306-8319-47b3-a81e-9e93cbfbfb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:41.599400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe4306-8319-47b3-a81e-9e93cbfbfb34 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.599679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe4306-8319-47b3-a81e-9e93cbfbfb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.599949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe4306-8319-47b3-a81e-9e93cbfbfb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.600921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe4306-8319-47b3-a81e-9e93cbfbfb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:41.603996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe4306-8319-47b3-a81e-9e93cbfbfb34 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:41.617601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1271/2508] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:03:41.621748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:41.631757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba90a-1c1d-4632-ace2-fcee0e6c6267 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:41.636402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:41.637040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.637354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.637627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] PATH_INFO: `/v3/projects/87edfa1673d2411dac1cd3a4f304406e/users/8e65f95e1caa4cfebae3b0471a7e871a/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.652623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=87edfa1673d2411dac1cd3a4f304406e, user_id=8e65f95e1caa4cfebae3b0471a7e871a, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:41.655340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:41.668714 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1238/2509] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:41 2022] PUT /identity/v3/projects/87edfa1673d2411dac1cd3a4f304406e/users/8e65f95e1caa4cfebae3b0471a7e871a/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:41.680241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861da7-9fd6-4220-93f0-c5834bc7a924 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:41.680531 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861da7-9fd6-4220-93f0-c5834bc7a924 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:41.680812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861da7-9fd6-4220-93f0-c5834bc7a924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:41.710757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6e861da7-9fd6-4220-93f0-c5834bc7a924 None None] MFA Rules not processed for user `8e65f95e1caa4cfebae3b0471a7e871a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:41.711990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6e861da7-9fd6-4220-93f0-c5834bc7a924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:41.917281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1272/2510] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:41.919195 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.001916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba150c-403a-4428-97f8-e535ea3c34ca admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.007489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.008108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.008389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.008661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.013880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.013880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.046658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1239/2511] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:41 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:42.050878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861da7-9fd6-4220-93f0-c5834bc7a924 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.067900 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861da7-9fd6-4220-93f0-c5834bc7a924 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.149653 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:42.154296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.155155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.155752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.155926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.157987 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.160736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.184688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1273/2512] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:42 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:42.189934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.201321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c906c6-bf15-48fc-b78b-30801c0c6823 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.206507 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.207198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.207524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.207894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.208645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.210559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.224270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1240/2513] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:42.229458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.241876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36fe90-3bc4-40e5-ab53-78c6b6de12a7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.247339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.247996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.248273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.248543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] PATH_INFO: `/v3/projects/c9eda47e76e14d0499d41c030cdcfc43/users/03eb59b898a04a4ab2c2d4f82b3e29b9/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.268024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9eda47e76e14d0499d41c030cdcfc43, user_id=03eb59b898a04a4ab2c2d4f82b3e29b9, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.270783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.288965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1274/2514] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:42 2022] PUT /identity/v3/projects/c9eda47e76e14d0499d41c030cdcfc43/users/03eb59b898a04a4ab2c2d4f82b3e29b9/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:42.290448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.301562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ee784-b027-4092-aa23-a7df4a651447 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.358243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ac5c5-9e94-47db-a873-736f5698a42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.358869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac5c5-9e94-47db-a873-736f5698a42b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.359662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac5c5-9e94-47db-a873-736f5698a42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.360349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac5c5-9e94-47db-a873-736f5698a42b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.360636 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ac5c5-9e94-47db-a873-736f5698a42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.362254 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ac5c5-9e94-47db-a873-736f5698a42b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.372623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1241/2515] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:42.377463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.387861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd0a12-d76c-46a1-bf4a-5b03dcfaf527 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.392601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.393272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.393551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.393818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] PATH_INFO: `/v3/projects/c9eda47e76e14d0499d41c030cdcfc43/users/03eb59b898a04a4ab2c2d4f82b3e29b9/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.395713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9eda47e76e14d0499d41c030cdcfc43, user_id=03eb59b898a04a4ab2c2d4f82b3e29b9, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.398438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.409338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1275/2516] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:42 2022] PUT /identity/v3/projects/c9eda47e76e14d0499d41c030cdcfc43/users/03eb59b898a04a4ab2c2d4f82b3e29b9/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:42.419910 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.420221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.420486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.450133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] MFA Rules not processed for user `03eb59b898a04a4ab2c2d4f82b3e29b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:42.451343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:42.483643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:42.520057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1242/2517] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:42.532503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.534208 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:42.596247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b677ac4-5098-4f8e-82cd-ea9721606ae0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.599985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.600640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.600943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.601289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.611083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.612460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.621905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1276/2518] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:42.699612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:42.700992 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:42.712860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223cb94-18be-42f5-8695-98a7ad86440a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:42.716458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5ec27-e451-47d6-94ce-eb18f02c9660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:42.717042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5ec27-e451-47d6-94ce-eb18f02c9660 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:42.717354 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5ec27-e451-47d6-94ce-eb18f02c9660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:42.717624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5ec27-e451-47d6-94ce-eb18f02c9660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:42.771981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5ec27-e451-47d6-94ce-eb18f02c9660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:42.773535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5ec27-e451-47d6-94ce-eb18f02c9660 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:42.783233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1243/2519] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:51.849350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:51.943444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1a524-81ba-469b-9b49-f1ca7829b9e6 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:51.949566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:51.950711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:51.951251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:51.951671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] PATH_INFO: `/v3/users/8e65f95e1caa4cfebae3b0471a7e871a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:51.953349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e65f95e1caa4cfebae3b0471a7e871a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:51.955503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:52.021497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e65f95e1caa4cfebae3b0471a7e871a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.029284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:52.032603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e65f95e1caa4cfebae3b0471a7e871a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.065238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e65f95e1caa4cfebae3b0471a7e871a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.066571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e65f95e1caa4cfebae3b0471a7e871a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.083453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e65f95e1caa4cfebae3b0471a7e871a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.096032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:52.096546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:52.096642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:52.099550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1277/2520] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:51 2022] DELETE /identity/v3/users/8e65f95e1caa4cfebae3b0471a7e871a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:52.122711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:52.124046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:52.208306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1244/2521] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:52.218782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:52.221348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:52.301942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b5c0f-cabc-4783-a9f6-a241492cf3db admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:52.306670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:52.307340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:52.307629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:52.307904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:52.318609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:52.320969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:52.338984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1278/2522] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:52.707259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:52.709962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:52.738030 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face94a-74cd-4f88-815c-293ab4263a4c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:52.744925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:52.745631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:52.745912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:52.746187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] PATH_INFO: `/v3/projects/87edfa1673d2411dac1cd3a4f304406e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:52.747562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=87edfa1673d2411dac1cd3a4f304406e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:52.749875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:52.799215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87edfa1673d2411dac1cd3a4f304406e', 'request_id': 'req-fb0345f6-311f-4a28-8bee-392e14064a8d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.805994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87edfa1673d2411dac1cd3a4f304406e', 'request_id': 'req-fb0345f6-311f-4a28-8bee-392e14064a8d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.812464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87edfa1673d2411dac1cd3a4f304406e', 'request_id': 'req-fb0345f6-311f-4a28-8bee-392e14064a8d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.814199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:52.826422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] The token cache is being invalidate because project 87edfa1673d2411dac1cd3a4f304406e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:52.826706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.827328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.829406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1245/2523] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:52 2022] DELETE /identity/v3/projects/87edfa1673d2411dac1cd3a4f304406e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:52.833516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:52.843523 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07060cb3-f34b-4da4-ad31-04ac5e4c9af0 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:52.899876 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:52.900557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:52.900761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:52.901054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] PATH_INFO: `/v3/users/03eb59b898a04a4ab2c2d4f82b3e29b9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:52.902199 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03eb59b898a04a4ab2c2d4f82b3e29b9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:52.904110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:52.948046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03eb59b898a04a4ab2c2d4f82b3e29b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.955936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03eb59b898a04a4ab2c2d4f82b3e29b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.967810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03eb59b898a04a4ab2c2d4f82b3e29b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.968411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03eb59b898a04a4ab2c2d4f82b3e29b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.970850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:52.976235 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f9745123-5009-44ea-bcbe-fc9e2ca60a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03eb59b898a04a4ab2c2d4f82b3e29b9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:52.983281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1279/2524] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:52 2022] DELETE /identity/v3/users/03eb59b898a04a4ab2c2d4f82b3e29b9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:53.275209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:53.278574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:53.306841 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0345f6-311f-4a28-8bee-392e14064a8d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:53.354953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ae860-0a07-420b-ab26-a77c4eac7cbd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.355309 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ae860-0a07-420b-ab26-a77c4eac7cbd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.355508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ae860-0a07-420b-ab26-a77c4eac7cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.376480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-df2ae860-0a07-420b-ab26-a77c4eac7cbd None None] MFA Rules not processed for user `9a50fbbf18584bb1a784283e768be7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:53.377720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-df2ae860-0a07-420b-ab26-a77c4eac7cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:53.380820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:53.382040 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.382496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.382984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] PATH_INFO: `/v3/projects/c9eda47e76e14d0499d41c030cdcfc43` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.384198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9eda47e76e14d0499d41c030cdcfc43)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:53.386163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:53.451657 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1280/2525] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:53.456444 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9eda47e76e14d0499d41c030cdcfc43', 'request_id': 'req-1729fb52-9302-40bc-9c3b-09c990123cb2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:53.467103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9eda47e76e14d0499d41c030cdcfc43', 'request_id': 'req-1729fb52-9302-40bc-9c3b-09c990123cb2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:53.475387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9eda47e76e14d0499d41c030cdcfc43', 'request_id': 'req-1729fb52-9302-40bc-9c3b-09c990123cb2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:53.489857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] The token cache is being invalidate because project c9eda47e76e14d0499d41c030cdcfc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:53.490146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:53.490783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1729fb52-9302-40bc-9c3b-09c990123cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:53.492779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1246/2526] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:53 2022] DELETE /identity/v3/projects/c9eda47e76e14d0499d41c030cdcfc43 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:53.497445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b3e3-d9bb-4bd0-ba25-6562489d58d5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.497838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b3e3-d9bb-4bd0-ba25-6562489d58d5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.498206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b3e3-d9bb-4bd0-ba25-6562489d58d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.518933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-27b9b3e3-d9bb-4bd0-ba25-6562489d58d5 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:53.520153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-27b9b3e3-d9bb-4bd0-ba25-6562489d58d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:53.591808 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1281/2527] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:53.599896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4ff0d-f4e2-4fc3-9548-1becbf043a46 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.600898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4ff0d-f4e2-4fc3-9548-1becbf043a46 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.602183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4ff0d-f4e2-4fc3-9548-1becbf043a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.628745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-70c4ff0d-f4e2-4fc3-9548-1becbf043a46 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:53.630106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-70c4ff0d-f4e2-4fc3-9548-1becbf043a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:53.641243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1247/2528] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:53.643164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9b3e3-d9bb-4bd0-ba25-6562489d58d5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:53.654937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9b3e3-d9bb-4bd0-ba25-6562489d58d5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:53.661171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:53.661772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.662044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.662313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.662952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:53.663247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:53.667161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:53.681452 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1282/2529] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:53.682876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4ff0d-f4e2-4fc3-9548-1becbf043a46 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:53.693622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4ff0d-f4e2-4fc3-9548-1becbf043a46 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:53.699202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:53.699499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.699852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.700204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.700896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:53.702891 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:53.749132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1248/2530] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:53 2022] POST /identity/v3/projects => generated 359 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:53.753341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:53.766527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372a2c-0a2e-4e99-86ed-f97add5a6fae admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:53.840308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:53.841738 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.842022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.842299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.842934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:53.845414 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:53.869670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1283/2531] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:53 2022] POST /identity/v3/users => generated 280 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:53.874096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:53.884351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577169-d9b4-4bea-9ab4-bb81d081f93b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:53.889620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:53.890221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.890496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.890761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.891406 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:53.892980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:53.903460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1249/2532] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:53.909885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:53.921665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f099ff5-44ac-4b98-a5b5-8822f8dffee2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:53.927993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5f538-8f1e-4592-b60f-8f544a91d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:53.928726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5f538-8f1e-4592-b60f-8f544a91d4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.929069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5f538-8f1e-4592-b60f-8f544a91d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.929524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5f538-8f1e-4592-b60f-8f544a91d4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.930232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5f538-8f1e-4592-b60f-8f544a91d4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:53.932858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5f538-8f1e-4592-b60f-8f544a91d4d4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:53.948856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1284/2533] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:53.953267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:53.964940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb241-3050-4f8b-9319-8803462e47ab admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:53.972607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:53.973331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:53.973731 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:53.974006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] PATH_INFO: `/v3/projects/ed4b03b4ab584665be661f3aa6072020/users/2c15a92fccde40da9f1537e4b5cc2aa9/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:53.996413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4b03b4ab584665be661f3aa6072020, user_id=2c15a92fccde40da9f1537e4b5cc2aa9, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:53.999233 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.023810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1250/2534] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:53 2022] PUT /identity/v3/projects/ed4b03b4ab584665be661f3aa6072020/users/2c15a92fccde40da9f1537e4b5cc2aa9/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:54.035097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36b60-4277-4af3-a267-f9e9bf138e45 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.035503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36b60-4277-4af3-a267-f9e9bf138e45 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.035939 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36b60-4277-4af3-a267-f9e9bf138e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.045751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:54.089815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-08e36b60-4277-4af3-a267-f9e9bf138e45 None None] MFA Rules not processed for user `2c15a92fccde40da9f1537e4b5cc2aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:54.091433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-08e36b60-4277-4af3-a267-f9e9bf138e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:54.191243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1285/2535] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:54.195699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.218671 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:54.281604 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138c089-2926-4e3f-8fa1-5550cd1b50e4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.287756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.288474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.288851 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.289214 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.290070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:54.292075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.318543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1251/2536] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:54 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:54.322812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e36b60-4277-4af3-a267-f9e9bf138e45 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.334304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e36b60-4277-4af3-a267-f9e9bf138e45 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.398851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.399480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.399761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.400213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.400686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:54.402650 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.426908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1286/2537] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:54 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:54.430880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.441618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64e4ca-dc08-4c2d-822a-ab2241c4a62c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.446372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.446961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.447238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.447501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.448127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:54.449746 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.461839 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1252/2538] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:54.465407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.475814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958ad79-e0f6-4bed-a2f3-69c228fb1701 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.480503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.481034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.481338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.481737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] PATH_INFO: `/v3/projects/757a5da014054b86a0173943e8089aa8/users/b853c04c842d409fa2b141fa0c72c11b/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.496468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757a5da014054b86a0173943e8089aa8, user_id=b853c04c842d409fa2b141fa0c72c11b, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:54.499399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.513297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1287/2539] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:54 2022] PUT /identity/v3/projects/757a5da014054b86a0173943e8089aa8/users/b853c04c842d409fa2b141fa0c72c11b/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:54.517544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.527932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55655d-4b62-41c1-8cd6-c4c348d2d96f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.591419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1ec27-dedb-4771-aa56-08fc2a0e37c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.592459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1ec27-dedb-4771-aa56-08fc2a0e37c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.592459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1ec27-dedb-4771-aa56-08fc2a0e37c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.592658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1ec27-dedb-4771-aa56-08fc2a0e37c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.593273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1ec27-dedb-4771-aa56-08fc2a0e37c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:54.594977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1ec27-dedb-4771-aa56-08fc2a0e37c1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.605272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1253/2540] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:54.609484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.619756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43683fc0-c3db-45bb-96e7-0f2dc3515664 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.624376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.625022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.625339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.625611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] PATH_INFO: `/v3/projects/757a5da014054b86a0173943e8089aa8/users/b853c04c842d409fa2b141fa0c72c11b/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.628054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] RBAC: Authorizing `identity:create_grant(project_id=757a5da014054b86a0173943e8089aa8, user_id=b853c04c842d409fa2b141fa0c72c11b, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:54.631169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.643631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1288/2541] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:54 2022] PUT /identity/v3/projects/757a5da014054b86a0173943e8089aa8/users/b853c04c842d409fa2b141fa0c72c11b/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:54.654997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c88c-1771-43c4-9e0c-42604d6ea8d4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.655283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c88c-1771-43c4-9e0c-42604d6ea8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.655555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c88c-1771-43c4-9e0c-42604d6ea8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.684200 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2028c88c-1771-43c4-9e0c-42604d6ea8d4 None None] MFA Rules not processed for user `b853c04c842d409fa2b141fa0c72c11b`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:54.685450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2028c88c-1771-43c4-9e0c-42604d6ea8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:54.760071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1254/2542] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:54.771801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.773505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:54.834602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:54.842137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e9cac-acb8-4411-814f-3a32808e057b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.846329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.847062 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.847351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.847626 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.857264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:54.858686 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:54.868924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1289/2543] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:54.958471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028c88c-1771-43c4-9e0c-42604d6ea8d4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:54.968579 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028c88c-1771-43c4-9e0c-42604d6ea8d4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:54.972834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:54.973525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:54.973781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:54.974052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:54.987555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:55.042218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:55.044711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:55.055496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1255/2544] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:03:56.256802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:56.258336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:56.342256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc708-7b0b-4179-94e8-627d2f416964 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:56.349129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:56.349983 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:56.350726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:56.353237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] PATH_INFO: `/v3/users/ccd1440631a94f7b87403dad65292579` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:56.358348 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd1440631a94f7b87403dad65292579)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:56.360920 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:56.444938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd1440631a94f7b87403dad65292579'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.459807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd1440631a94f7b87403dad65292579'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.477125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd1440631a94f7b87403dad65292579'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.477995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd1440631a94f7b87403dad65292579'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.488769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd1440631a94f7b87403dad65292579'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.496619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1290/2545] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:56 2022] DELETE /identity/v3/users/ccd1440631a94f7b87403dad65292579 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:56.778827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:56.803849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041c4d7-7728-4ff4-bf21-4608b35e3f46 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:56.872094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:56.872724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:56.873000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:56.873326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] PATH_INFO: `/v3/projects/f970622440bb427382b99e4a68d8746e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:56.874504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f970622440bb427382b99e4a68d8746e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:56.876380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:56.933473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f970622440bb427382b99e4a68d8746e', 'request_id': 'req-b165637a-9f43-4bca-acb1-c3b3cf96301b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.954373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f970622440bb427382b99e4a68d8746e', 'request_id': 'req-b165637a-9f43-4bca-acb1-c3b3cf96301b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.966713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f970622440bb427382b99e4a68d8746e', 'request_id': 'req-b165637a-9f43-4bca-acb1-c3b3cf96301b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.989667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] The token cache is being invalidate because project f970622440bb427382b99e4a68d8746e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:56.989959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.990648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:56.992197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1256/2546] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:56 2022] DELETE /identity/v3/projects/f970622440bb427382b99e4a68d8746e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:56.996123 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:56.997572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:57.010863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303e01-0403-406b-96f2-63fde4f6c648 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.071962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.072601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.072877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.073146 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] PATH_INFO: `/v3/users/daf69c0483224a3bb2b9aefe96e6d458` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.074317 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf69c0483224a3bb2b9aefe96e6d458)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:57.076249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:57.087593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:57.134288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf69c0483224a3bb2b9aefe96e6d458'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.143179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf69c0483224a3bb2b9aefe96e6d458'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.157861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf69c0483224a3bb2b9aefe96e6d458'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.159198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf69c0483224a3bb2b9aefe96e6d458'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.168111 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-250f4036-126c-4aab-9b3a-58f3fe314f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf69c0483224a3bb2b9aefe96e6d458'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.174782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1291/2547] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:56 2022] DELETE /identity/v3/users/daf69c0483224a3bb2b9aefe96e6d458 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:57.350519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:57.352424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:57.380302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165637a-9f43-4bca-acb1-c3b3cf96301b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.429852 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:57.449117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.449894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.450275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.450552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] PATH_INFO: `/v3/projects/9b3f3b15aa404ddfb641fe254765dcb8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.451719 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b3f3b15aa404ddfb641fe254765dcb8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:57.454502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:57.506087 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b3f3b15aa404ddfb641fe254765dcb8', 'request_id': 'req-131148f1-7f56-4d25-94d2-7e248c1754b0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.512464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b3f3b15aa404ddfb641fe254765dcb8', 'request_id': 'req-131148f1-7f56-4d25-94d2-7e248c1754b0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.518875 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b3f3b15aa404ddfb641fe254765dcb8', 'request_id': 'req-131148f1-7f56-4d25-94d2-7e248c1754b0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.532965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] The token cache is being invalidate because project 9b3f3b15aa404ddfb641fe254765dcb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:57.533291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.533971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-131148f1-7f56-4d25-94d2-7e248c1754b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:57.535553 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1257/2548] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:57 2022] DELETE /identity/v3/projects/9b3f3b15aa404ddfb641fe254765dcb8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:57.540410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956d43-63fb-4967-88eb-3bd23cf057c4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.540880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956d43-63fb-4967-88eb-3bd23cf057c4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.540978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956d43-63fb-4967-88eb-3bd23cf057c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.560434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-43956d43-63fb-4967-88eb-3bd23cf057c4 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:57.561681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-43956d43-63fb-4967-88eb-3bd23cf057c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:57.627884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1292/2549] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:57.636004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c021-9462-4d1c-81bb-4fc096f83c00 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.636311 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c021-9462-4d1c-81bb-4fc096f83c00 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.636594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c021-9462-4d1c-81bb-4fc096f83c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.658408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9930c021-9462-4d1c-81bb-4fc096f83c00 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:57.659643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9930c021-9462-4d1c-81bb-4fc096f83c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:57.668029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1258/2550] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:03:57.672370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956d43-63fb-4967-88eb-3bd23cf057c4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:57.682552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956d43-63fb-4967-88eb-3bd23cf057c4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.687389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.688011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.689007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.689313 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.689977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:57.690282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:57.691918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:57.705050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1293/2551] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:57.709562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c021-9462-4d1c-81bb-4fc096f83c00 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:57.719651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c021-9462-4d1c-81bb-4fc096f83c00 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.724457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.725253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.725538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.725815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.726470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:57.728515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:57.765026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1259/2552] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:57 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:03:57.767128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:57.778204 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74db29-036b-413c-815d-d06cbea07cdb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.859017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.859625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.859899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.860702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.861362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:57.863263 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:57.891916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1294/2553] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:57 2022] POST /identity/v3/users => generated 281 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:57.897470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:57.909109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fccbf0-968e-4ee2-9b7c-25e61637ad85 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.914573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.915337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.915623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.915906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.916665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:57.918729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:57.930465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1260/2554] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:57.934549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:57.942163 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:57.946337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001150-3b32-493c-b589-0b3d808eda41 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.954610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc994e47-9f23-4f00-828b-f52686be882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.955628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc994e47-9f23-4f00-828b-f52686be882c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.956384 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc994e47-9f23-4f00-828b-f52686be882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.956909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc994e47-9f23-4f00-828b-f52686be882c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:57.957812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc994e47-9f23-4f00-828b-f52686be882c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:57.959649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc994e47-9f23-4f00-828b-f52686be882c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:57.972767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1295/2555] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:03:57.976623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:57.991092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065759ba-8dda-45b7-8935-5692450ec5f8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:57.996938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:57.997898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:57.998285 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:57.998641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] PATH_INFO: `/v3/projects/8e11b13cf63840bb82532b5e6bc03a1d/users/e9d059a2d9254450938b8c11a9677a7b/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:58.036823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e11b13cf63840bb82532b5e6bc03a1d, user_id=e9d059a2d9254450938b8c11a9677a7b, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:58.040528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:58.085959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1261/2556] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:57 2022] PUT /identity/v3/projects/8e11b13cf63840bb82532b5e6bc03a1d/users/e9d059a2d9254450938b8c11a9677a7b/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:03:58.150304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833f77e-1052-4c0a-b657-73fb392b1f3c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:58.150938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833f77e-1052-4c0a-b657-73fb392b1f3c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:58.151411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833f77e-1052-4c0a-b657-73fb392b1f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:58.187336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7833f77e-1052-4c0a-b657-73fb392b1f3c None None] MFA Rules not processed for user `e9d059a2d9254450938b8c11a9677a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:58.189720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7833f77e-1052-4c0a-b657-73fb392b1f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:58.266306 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1296/2557] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:58.308405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:58.310172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:58.377778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531efd58-2317-4c87-b498-760c70a26e5c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:58.381825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9466d82-8cc4-447c-883c-6ad3f983cd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:58.382447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9466d82-8cc4-447c-883c-6ad3f983cd3b service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:58.382732 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9466d82-8cc4-447c-883c-6ad3f983cd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:58.383003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9466d82-8cc4-447c-883c-6ad3f983cd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:58.391563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9466d82-8cc4-447c-883c-6ad3f983cd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:58.392989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9466d82-8cc4-447c-883c-6ad3f983cd3b service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:58.402717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1262/2558] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:58.419302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833f77e-1052-4c0a-b657-73fb392b1f3c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:58.493482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833f77e-1052-4c0a-b657-73fb392b1f3c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:58.498178 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:58.498965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:58.499594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:58.499703 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] PATH_INFO: `/v3/users/e9d059a2d9254450938b8c11a9677a7b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:58.500922 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9d059a2d9254450938b8c11a9677a7b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:58.502961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:58.555837 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9d059a2d9254450938b8c11a9677a7b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:58.563442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9d059a2d9254450938b8c11a9677a7b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:58.577075 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9d059a2d9254450938b8c11a9677a7b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:58.577835 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9d059a2d9254450938b8c11a9677a7b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:58.586423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9d059a2d9254450938b8c11a9677a7b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:58.592593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1297/2559] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:03:58 2022] DELETE /identity/v3/users/e9d059a2d9254450938b8c11a9677a7b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:58.597916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:58.598202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:58.598473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:58.617025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:58.618249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:58.623127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:58.678050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1263/2560] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:58.687719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:58.689149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:58.694631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:58.748262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87c695-bb1b-47d9-8af6-e4d1e66880ee admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:58.751966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114171ff-d48e-4f42-8dad-b1c146471ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:58.752558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114171ff-d48e-4f42-8dad-b1c146471ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:58.752906 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114171ff-d48e-4f42-8dad-b1c146471ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:58.753192 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114171ff-d48e-4f42-8dad-b1c146471ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:58.761917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114171ff-d48e-4f42-8dad-b1c146471ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:58.763328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114171ff-d48e-4f42-8dad-b1c146471ef2 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:58.772990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1298/2561] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:03:59.040884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.042416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:59.057055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992edcdf-7e69-4af2-a593-1155240b21b2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.062689 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.063687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.063938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.064201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] PATH_INFO: `/v3/projects/8e11b13cf63840bb82532b5e6bc03a1d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.065345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e11b13cf63840bb82532b5e6bc03a1d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.067573 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.114385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e11b13cf63840bb82532b5e6bc03a1d', 'request_id': 'req-129ce0b8-3814-4dff-8b4e-2df862e19682'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:59.121119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e11b13cf63840bb82532b5e6bc03a1d', 'request_id': 'req-129ce0b8-3814-4dff-8b4e-2df862e19682'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:59.127251 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e11b13cf63840bb82532b5e6bc03a1d', 'request_id': 'req-129ce0b8-3814-4dff-8b4e-2df862e19682'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:59.139681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] The token cache is being invalidate because project 8e11b13cf63840bb82532b5e6bc03a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:03:59.139977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:59.140591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-129ce0b8-3814-4dff-8b4e-2df862e19682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:03:59.142729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1264/2562] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:03:59 2022] DELETE /identity/v3/projects/8e11b13cf63840bb82532b5e6bc03a1d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:59.154341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6fe63-71f2-4500-8238-46d7d2a112ee None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.154341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6fe63-71f2-4500-8238-46d7d2a112ee None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.154341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6fe63-71f2-4500-8238-46d7d2a112ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.167699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a7f6fe63-71f2-4500-8238-46d7d2a112ee None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:59.168880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a7f6fe63-71f2-4500-8238-46d7d2a112ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:59.226181 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1299/2563] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:59.233806 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844db51-f1cb-41af-b116-ac12171a3b47 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.234091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844db51-f1cb-41af-b116-ac12171a3b47 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.234363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844db51-f1cb-41af-b116-ac12171a3b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.255423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1844db51-f1cb-41af-b116-ac12171a3b47 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:59.256815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1844db51-f1cb-41af-b116-ac12171a3b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:59.268647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1265/2564] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:59.269632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6fe63-71f2-4500-8238-46d7d2a112ee None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.279047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6fe63-71f2-4500-8238-46d7d2a112ee None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.283949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.284538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.284815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.285085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.285783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:03:59.286088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.287683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.299890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1300/2565] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:03:59.304144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844db51-f1cb-41af-b116-ac12171a3b47 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.313882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844db51-f1cb-41af-b116-ac12171a3b47 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.319141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.319747 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.320022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.320294 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.320934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.322877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.352995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1266/2566] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:03:59 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:59.357409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.367798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb866d18-1c11-4f01-8980-5cc643469f6b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.427895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.428492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.428768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.429039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.429705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.431600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.454350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1301/2567] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:03:59 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:03:59.459422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.470077 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6975464-1733-4d4a-a980-e0d35f13e614 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.474692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.475288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.475563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.475832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.476518 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.478070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.488876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1267/2568] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:59.493132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.503534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0250-57d7-4da4-811b-deeee721ef69 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.508087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f43ff-8f8a-4a2e-9c03-cae3e8fa156b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.508668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f43ff-8f8a-4a2e-9c03-cae3e8fa156b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.508944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f43ff-8f8a-4a2e-9c03-cae3e8fa156b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.509230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f43ff-8f8a-4a2e-9c03-cae3e8fa156b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.509862 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f43ff-8f8a-4a2e-9c03-cae3e8fa156b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.511434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f43ff-8f8a-4a2e-9c03-cae3e8fa156b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.522112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1302/2569] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:03:59.526120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.536612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c954a-2c75-46a6-a4e1-4932b2f65e61 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.541829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.542645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.543010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.543371 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] PATH_INFO: `/v3/projects/6b4c6214b457477e875a59ba567a24bb/users/3f1236413c5f4da4aa440e6be0c4b9a0/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.559554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4c6214b457477e875a59ba567a24bb, user_id=3f1236413c5f4da4aa440e6be0c4b9a0, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.562325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.580536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1268/2570] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:03:59 2022] PUT /identity/v3/projects/6b4c6214b457477e875a59ba567a24bb/users/3f1236413c5f4da4aa440e6be0c4b9a0/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:03:59.592582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038e6b-36da-40ed-886b-b4640e7ba792 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.593117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038e6b-36da-40ed-886b-b4640e7ba792 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.593117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038e6b-36da-40ed-886b-b4640e7ba792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.613747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:59.622032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-16038e6b-36da-40ed-886b-b4640e7ba792 None None] MFA Rules not processed for user `3f1236413c5f4da4aa440e6be0c4b9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:03:59.623583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16038e6b-36da-40ed-886b-b4640e7ba792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:59.684927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1303/2571] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:03:59.696748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:03:59.698434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:03:59.731566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:03:59.755672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc684b-be74-415d-9a71-84a64e159e2e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:03:59.759405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92101af2-b29e-45f7-8266-447856c00b77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:03:59.759997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92101af2-b29e-45f7-8266-447856c00b77 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:03:59.760270 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92101af2-b29e-45f7-8266-447856c00b77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:03:59.760536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92101af2-b29e-45f7-8266-447856c00b77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:03:59.768606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92101af2-b29e-45f7-8266-447856c00b77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:03:59.770052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92101af2-b29e-45f7-8266-447856c00b77 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:03:59.780298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1269/2572] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:03:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:03.393464 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038e6b-36da-40ed-886b-b4640e7ba792 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:03.461457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038e6b-36da-40ed-886b-b4640e7ba792 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:03.466969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:03.467586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:03.467861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:03.468150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] PATH_INFO: `/v3/users/3f1236413c5f4da4aa440e6be0c4b9a0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:03.469399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1236413c5f4da4aa440e6be0c4b9a0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:03.471379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:03.517954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1236413c5f4da4aa440e6be0c4b9a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:03.525937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1236413c5f4da4aa440e6be0c4b9a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:03.539636 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1236413c5f4da4aa440e6be0c4b9a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:03.540339 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1236413c5f4da4aa440e6be0c4b9a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:03.548074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1236413c5f4da4aa440e6be0c4b9a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:03.554953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1304/2573] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:03 2022] DELETE /identity/v3/users/3f1236413c5f4da4aa440e6be0c4b9a0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:03.559424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:03.559921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:03.560021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:03.581741 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:03.582979 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:03.645068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1270/2574] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:03.654722 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:03.656440 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:03.677639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:03.715254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ae739-aba1-4d37-b29c-29106e58984b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:03.718953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:03.719612 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:03.719841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:03.720114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:03.729165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:03.730619 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:03.740479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1305/2575] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:04.013810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:04.015256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:04.031262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cffd0-6922-4dff-a08a-20367d6ec97c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:04.036460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:04.037102 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:04.037413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:04.037682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] PATH_INFO: `/v3/projects/6b4c6214b457477e875a59ba567a24bb` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:04.038810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4c6214b457477e875a59ba567a24bb)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:04.041879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:04.114133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4c6214b457477e875a59ba567a24bb', 'request_id': 'req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.123426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4c6214b457477e875a59ba567a24bb', 'request_id': 'req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.130025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4c6214b457477e875a59ba567a24bb', 'request_id': 'req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.144767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] The token cache is being invalidate because project 6b4c6214b457477e875a59ba567a24bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:04.145065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.145734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4b30f0c-8a00-46d4-9b47-88f8d52e9531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.148213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1271/2576] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:04 2022] DELETE /identity/v3/projects/6b4c6214b457477e875a59ba567a24bb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:04.578363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:04.580327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:04.595213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd23659-e8fd-4a43-98c9-2041f7d0dcfa service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:04.650807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:04.651459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:04.651735 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:04.652005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] PATH_INFO: `/v3/users/2c15a92fccde40da9f1537e4b5cc2aa9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:04.653147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c15a92fccde40da9f1537e4b5cc2aa9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:04.655085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:04.700311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c15a92fccde40da9f1537e4b5cc2aa9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.707654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c15a92fccde40da9f1537e4b5cc2aa9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.719232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c15a92fccde40da9f1537e4b5cc2aa9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.719888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c15a92fccde40da9f1537e4b5cc2aa9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.727587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c15a92fccde40da9f1537e4b5cc2aa9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:04.733784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1306/2577] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:04 2022] DELETE /identity/v3/users/2c15a92fccde40da9f1537e4b5cc2aa9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:04.738557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:04.739014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:04.739111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:04.758028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:04.759203 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:04.807935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:04.817846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1272/2578] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:04.827328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:04.828765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:04.874446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:04.887398 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78c119-a761-4e45-b6ee-2ccdcd67d133 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:04.891137 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:04.891733 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:04.892494 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:04.892766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:04.901685 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:04.903098 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:04.913135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1307/2579] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:05.188113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:05.190198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:05.208949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b86b5-3287-4a69-858d-4e8d7b5b5831 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:05.213562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:05.214193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:05.214471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:05.214740 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] PATH_INFO: `/v3/projects/ed4b03b4ab584665be661f3aa6072020` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:05.217309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4b03b4ab584665be661f3aa6072020)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:05.218464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:05.270769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4b03b4ab584665be661f3aa6072020', 'request_id': 'req-bb7b5801-94d3-4d34-99d6-1d2683e69613'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.277657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4b03b4ab584665be661f3aa6072020', 'request_id': 'req-bb7b5801-94d3-4d34-99d6-1d2683e69613'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.284958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4b03b4ab584665be661f3aa6072020', 'request_id': 'req-bb7b5801-94d3-4d34-99d6-1d2683e69613'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.306538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] The token cache is being invalidate because project ed4b03b4ab584665be661f3aa6072020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:05.306838 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.307961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.309883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1273/2580] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:05 2022] DELETE /identity/v3/projects/ed4b03b4ab584665be661f3aa6072020 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:05.313770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:05.324574 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9a4e-df02-48ff-94c7-14c495468900 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:05.383664 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:05.384294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:05.384573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:05.384845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] PATH_INFO: `/v3/users/b853c04c842d409fa2b141fa0c72c11b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:05.386012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b853c04c842d409fa2b141fa0c72c11b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:05.388112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:05.432571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b853c04c842d409fa2b141fa0c72c11b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.439997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b853c04c842d409fa2b141fa0c72c11b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.451632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b853c04c842d409fa2b141fa0c72c11b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.452307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b853c04c842d409fa2b141fa0c72c11b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.459528 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ded9ff8-65ca-411b-b53f-a16a8199d626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b853c04c842d409fa2b141fa0c72c11b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.465749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1308/2581] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:05 2022] DELETE /identity/v3/users/b853c04c842d409fa2b141fa0c72c11b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:05.739545 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:05.741441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:05.763549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b5801-94d3-4d34-99d6-1d2683e69613 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:05.832761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:05.833427 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:05.833775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:05.834048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] PATH_INFO: `/v3/projects/757a5da014054b86a0173943e8089aa8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:05.835179 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757a5da014054b86a0173943e8089aa8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:05.837055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:05.884578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757a5da014054b86a0173943e8089aa8', 'request_id': 'req-9796c120-600f-466f-aa2c-eae0cef9a0d7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.891011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757a5da014054b86a0173943e8089aa8', 'request_id': 'req-9796c120-600f-466f-aa2c-eae0cef9a0d7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.896449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757a5da014054b86a0173943e8089aa8', 'request_id': 'req-9796c120-600f-466f-aa2c-eae0cef9a0d7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.909813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] The token cache is being invalidate because project 757a5da014054b86a0173943e8089aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:05.910105 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.910758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9796c120-600f-466f-aa2c-eae0cef9a0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:05.912372 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1274/2582] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:05 2022] DELETE /identity/v3/projects/757a5da014054b86a0173943e8089aa8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:05.919043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed27a1-38cf-4831-afbb-20eed9c007b1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:05.919521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed27a1-38cf-4831-afbb-20eed9c007b1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:05.919616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed27a1-38cf-4831-afbb-20eed9c007b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:05.939809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9aed27a1-38cf-4831-afbb-20eed9c007b1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:05.941125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9aed27a1-38cf-4831-afbb-20eed9c007b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:06.000427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1309/2583] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:06.008347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14c56-753c-4ce2-9a8b-45b946168006 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.008633 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14c56-753c-4ce2-9a8b-45b946168006 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.008902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14c56-753c-4ce2-9a8b-45b946168006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.030511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-75b14c56-753c-4ce2-9a8b-45b946168006 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:06.031958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-75b14c56-753c-4ce2-9a8b-45b946168006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:06.040770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1275/2584] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:06.045057 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed27a1-38cf-4831-afbb-20eed9c007b1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.051908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:06.056825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed27a1-38cf-4831-afbb-20eed9c007b1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.063023 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.064028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.065031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.065352 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.065992 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:06.066302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.067902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.080947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1310/2585] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:06.085605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14c56-753c-4ce2-9a8b-45b946168006 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.094181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:06.100514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14c56-753c-4ce2-9a8b-45b946168006 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.107744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.108583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.109213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.109522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.110392 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.112493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.142517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1276/2586] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:06 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:06.146742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.159954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56eabf7-515d-422d-8ce1-15b25f2115c1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.224597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.225231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.225519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.225789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.226423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.228693 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.250910 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1311/2587] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:06 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:06.255106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.265879 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8807-adae-4923-b692-8fcba81f827b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.271562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.272154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.272435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.272716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.273393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.274973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.286380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1277/2588] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:06.290226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.300382 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd07d21-8e4b-4b17-a59d-762b1011187b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.305495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f232d-12be-4668-b164-b5ed73a20c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.306081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f232d-12be-4668-b164-b5ed73a20c68 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.306357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f232d-12be-4668-b164-b5ed73a20c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.306625 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f232d-12be-4668-b164-b5ed73a20c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.307255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f232d-12be-4668-b164-b5ed73a20c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.308853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f232d-12be-4668-b164-b5ed73a20c68 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.319210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1312/2589] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:06.323260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.333876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81690be5-675a-4018-8a36-60752e3c12b7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.338373 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.339010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.339289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.339557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] PATH_INFO: `/v3/projects/3eed6d8acd5c42fbbfa705c4207eddb5/users/f0799fca35304c41920085818220aa07/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.354584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eed6d8acd5c42fbbfa705c4207eddb5, user_id=f0799fca35304c41920085818220aa07, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.357308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.371423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1278/2590] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:06 2022] PUT /identity/v3/projects/3eed6d8acd5c42fbbfa705c4207eddb5/users/f0799fca35304c41920085818220aa07/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:06.381811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5b59d-33a5-4596-a3ce-08b78dbfc695 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.382094 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5b59d-33a5-4596-a3ce-08b78dbfc695 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.382362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5b59d-33a5-4596-a3ce-08b78dbfc695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.416665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-40d5b59d-33a5-4596-a3ce-08b78dbfc695 None None] MFA Rules not processed for user `f0799fca35304c41920085818220aa07`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:06.418310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-40d5b59d-33a5-4596-a3ce-08b78dbfc695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:06.503026 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1313/2591] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:06.507683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.583638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7653019-db1c-4cd9-8f16-a1162ff2b30f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.591531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.592484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.592946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.593249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.594052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.595975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.628609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1279/2592] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:06 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:06.630001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5b59d-33a5-4596-a3ce-08b78dbfc695 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.643120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5b59d-33a5-4596-a3ce-08b78dbfc695 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.701262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.701976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.702318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.702678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.703387 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.705962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.714175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:06.727894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1314/2593] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:06 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:06.732412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.743014 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ffdea-efc1-452b-9697-8c13db23f506 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.748424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.749025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.749347 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.749622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.750264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.752389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.763198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1280/2594] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:06.767748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.777806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90252d5a-ba07-431d-990f-a14830b9a666 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.783003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e73f1-c869-4da6-9819-778218ce8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.783601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e73f1-c869-4da6-9819-778218ce8ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.783903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e73f1-c869-4da6-9819-778218ce8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.784173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e73f1-c869-4da6-9819-778218ce8ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.784809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e73f1-c869-4da6-9819-778218ce8ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.786467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e73f1-c869-4da6-9819-778218ce8ce8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.796632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1315/2595] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:06.801257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:06.811044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01400750-4303-4d13-9c8c-61b3ee1b772a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:06.816363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:06.817013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.817321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.817594 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] PATH_INFO: `/v3/projects/c506af1be1ab49d0b1f7c5b40de682b5/users/8bb125b7a23c41feb4c23976f867c642/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.832287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c506af1be1ab49d0b1f7c5b40de682b5, user_id=8bb125b7a23c41feb4c23976f867c642, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:06.834822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:06.846950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1281/2596] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:06 2022] PUT /identity/v3/projects/c506af1be1ab49d0b1f7c5b40de682b5/users/8bb125b7a23c41feb4c23976f867c642/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:06.911826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25ee62-d7d9-4c9d-95e1-f56dab0e399d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:06.912108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25ee62-d7d9-4c9d-95e1-f56dab0e399d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:06.912380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25ee62-d7d9-4c9d-95e1-f56dab0e399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:06.940911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1b25ee62-d7d9-4c9d-95e1-f56dab0e399d None None] MFA Rules not processed for user `8bb125b7a23c41feb4c23976f867c642`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:06.942195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1b25ee62-d7d9-4c9d-95e1-f56dab0e399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:07.004393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1316/2597] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:07.008832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:07.068621 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:07.072434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb8b78-c4e8-4f3f-a861-1dd11f625c2a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:07.077984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:07.078587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.078867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.079141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.079817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:07.081745 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:07.107644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1282/2598] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:07 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:07.111969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25ee62-d7d9-4c9d-95e1-f56dab0e399d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:07.122402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25ee62-d7d9-4c9d-95e1-f56dab0e399d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:07.179226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:07.179827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.180108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.180383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.180998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:07.182955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:07.203813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1317/2599] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:07 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:07.207896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:07.217999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a78c2-461d-477d-a25e-06b15a419777 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:07.223023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:07.223606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.223882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.224147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.224771 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:07.226381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:07.237153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1283/2600] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:07.241409 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:07.251013 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af707e-c9da-45ef-a62b-db28e5ca67e1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:07.255598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:07.256231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.256511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.256782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] PATH_INFO: `/v3/projects/ff3541aae3f544c2927d89d1a1d9fe8f/users/2b04a55ddb3540c58c20a23173ee9660/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.270882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3541aae3f544c2927d89d1a1d9fe8f, user_id=2b04a55ddb3540c58c20a23173ee9660, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:07.273564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:07.286909 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1318/2601] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:07 2022] PUT /identity/v3/projects/ff3541aae3f544c2927d89d1a1d9fe8f/users/2b04a55ddb3540c58c20a23173ee9660/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:07.290751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:07.301931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b5362-5c92-4942-83ee-046095458a98 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:07.360273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98699da7-0483-4f66-8ba6-4cacc5e6faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:07.361447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98699da7-0483-4f66-8ba6-4cacc5e6faec admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.361729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98699da7-0483-4f66-8ba6-4cacc5e6faec admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.362000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98699da7-0483-4f66-8ba6-4cacc5e6faec admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.362644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98699da7-0483-4f66-8ba6-4cacc5e6faec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:07.364239 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98699da7-0483-4f66-8ba6-4cacc5e6faec admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:07.374197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1284/2602] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:07.378351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:07.388640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafa6eb-053d-4453-8abc-3c7e61e994b2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:07.393406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:07.394049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.394329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.394602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] PATH_INFO: `/v3/projects/ff3541aae3f544c2927d89d1a1d9fe8f/users/2b04a55ddb3540c58c20a23173ee9660/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.396510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3541aae3f544c2927d89d1a1d9fe8f, user_id=2b04a55ddb3540c58c20a23173ee9660, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:07.399213 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:07.410595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1319/2603] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:07 2022] PUT /identity/v3/projects/ff3541aae3f544c2927d89d1a1d9fe8f/users/2b04a55ddb3540c58c20a23173ee9660/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:07.420685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.420969 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.421267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.449883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] MFA Rules not processed for user `2b04a55ddb3540c58c20a23173ee9660`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:07.451102 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:07.520772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1285/2604] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:07.532358 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:07.534575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:07.558896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:07.598360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a36be3-5332-40fd-8cde-98fbc5c6dce6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:07.602668 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:07.603279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:07.603621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:07.603844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:07.678907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:07.680388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:07.689953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1320/2605] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:09.206378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:09.207778 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:09.219644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db9f15-87ed-4005-8d8e-258c88c46f6e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:09.223829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:09.224424 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:09.224700 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:09.224967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:09.270911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:09.280664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:09.282360 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:09.291569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1286/2606] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:09.828101 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:09.837346 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4013b0-6f0a-4247-9666-4501730b488a service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:09.840926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009afc3-f37b-440f-887f-9a2bd21a39a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:09.841550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009afc3-f37b-440f-887f-9a2bd21a39a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:09.841928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009afc3-f37b-440f-887f-9a2bd21a39a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:09.842207 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009afc3-f37b-440f-887f-9a2bd21a39a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:09.853890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009afc3-f37b-440f-887f-9a2bd21a39a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:09.855356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009afc3-f37b-440f-887f-9a2bd21a39a4 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:09.867107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1321/2607] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:14.068482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:14.150715 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b579289-43e0-4666-ba21-19de368cc007 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:14.155557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:14.156446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:14.156723 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:14.157110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] PATH_INFO: `/v3/users/f0799fca35304c41920085818220aa07` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:14.158767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0799fca35304c41920085818220aa07)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:14.160659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:14.211907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0799fca35304c41920085818220aa07'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.224543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0799fca35304c41920085818220aa07'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.231622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0799fca35304c41920085818220aa07'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.239619 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0799fca35304c41920085818220aa07'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.240293 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0799fca35304c41920085818220aa07'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.246342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1287/2608] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:14 2022] DELETE /identity/v3/users/f0799fca35304c41920085818220aa07 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:14.250896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:14.251402 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:14.251521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:14.270012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:14.271211 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:14.331161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1322/2609] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:14.340890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:14.342477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:14.403175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbee72-c807-446b-b867-ec442cf0cf20 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:14.407036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:14.408192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:14.408466 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:14.408729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:14.417765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:14.419180 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:14.428918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1288/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:14.745880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:14.747403 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:14.765803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8ca70-89c5-4cdb-8e1b-86fefbb51674 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:14.774617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:14.775360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:14.775590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:14.775855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] PATH_INFO: `/v3/projects/3eed6d8acd5c42fbbfa705c4207eddb5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:14.777404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eed6d8acd5c42fbbfa705c4207eddb5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:14.780437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:14.827984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eed6d8acd5c42fbbfa705c4207eddb5', 'request_id': 'req-21157eb2-61f3-4f9e-8a35-39a200a17098'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.834915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eed6d8acd5c42fbbfa705c4207eddb5', 'request_id': 'req-21157eb2-61f3-4f9e-8a35-39a200a17098'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.837223 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:14.843070 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eed6d8acd5c42fbbfa705c4207eddb5', 'request_id': 'req-21157eb2-61f3-4f9e-8a35-39a200a17098'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.855616 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] The token cache is being invalidate because project 3eed6d8acd5c42fbbfa705c4207eddb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:14.856076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.856783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:14.858355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1323/2611] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:14 2022] DELETE /identity/v3/projects/3eed6d8acd5c42fbbfa705c4207eddb5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:14.862530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:14.872435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209b6dd-bd8a-41a5-bd8a-08afb7cc469c service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:14.882649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:14.952519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:14.952519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:14.952519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:14.952519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] PATH_INFO: `/v3/users/8bb125b7a23c41feb4c23976f867c642` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:14.958156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb125b7a23c41feb4c23976f867c642)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:14.960438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:15.014670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb125b7a23c41feb4c23976f867c642'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.030225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb125b7a23c41feb4c23976f867c642'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.039278 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb125b7a23c41feb4c23976f867c642'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.048906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb125b7a23c41feb4c23976f867c642'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.049834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb125b7a23c41feb4c23976f867c642'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.057442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1289/2612] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:14 2022] DELETE /identity/v3/users/8bb125b7a23c41feb4c23976f867c642 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:15.762867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:15.765521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:15.794050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21157eb2-61f3-4f9e-8a35-39a200a17098 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:15.862783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:15.863323 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:15.863604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:15.863879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] PATH_INFO: `/v3/projects/c506af1be1ab49d0b1f7c5b40de682b5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:15.864993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c506af1be1ab49d0b1f7c5b40de682b5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:15.866940 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:15.917801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c506af1be1ab49d0b1f7c5b40de682b5', 'request_id': 'req-e3bc14e1-e7e9-4588-b8b2-086a264b8876'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.924735 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c506af1be1ab49d0b1f7c5b40de682b5', 'request_id': 'req-e3bc14e1-e7e9-4588-b8b2-086a264b8876'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.930782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c506af1be1ab49d0b1f7c5b40de682b5', 'request_id': 'req-e3bc14e1-e7e9-4588-b8b2-086a264b8876'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.944957 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] The token cache is being invalidate because project c506af1be1ab49d0b1f7c5b40de682b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:15.945274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.945897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:15.947530 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1324/2613] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:15 2022] DELETE /identity/v3/projects/c506af1be1ab49d0b1f7c5b40de682b5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:15.951559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:15.953499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:15.969496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399cac0-5c09-47d2-9602-20a99346e1e2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:16.036221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:16.036971 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:16.037280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:16.037644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] PATH_INFO: `/v3/users/2b04a55ddb3540c58c20a23173ee9660` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:16.038904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b04a55ddb3540c58c20a23173ee9660)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:16.040899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:16.085523 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:16.097044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b04a55ddb3540c58c20a23173ee9660'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.110096 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b04a55ddb3540c58c20a23173ee9660'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.117394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b04a55ddb3540c58c20a23173ee9660'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.127065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b04a55ddb3540c58c20a23173ee9660'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.127691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b04a55ddb3540c58c20a23173ee9660'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.135150 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1290/2614] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:15 2022] DELETE /identity/v3/users/2b04a55ddb3540c58c20a23173ee9660 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:16.250386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:16.252760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:16.318608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc14e1-e7e9-4588-b8b2-086a264b8876 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:16.348556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:16.371554 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e7567-5ee9-4fd6-a47b-06baf42ed7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:16.372229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e7567-5ee9-4fd6-a47b-06baf42ed7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:16.372577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e7567-5ee9-4fd6-a47b-06baf42ed7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:16.372917 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e7567-5ee9-4fd6-a47b-06baf42ed7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:16.374937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ee8e7567-5ee9-4fd6-a47b-06baf42ed7b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:16.441279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e7567-5ee9-4fd6-a47b-06baf42ed7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:16.442725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e7567-5ee9-4fd6-a47b-06baf42ed7b8 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:16.452869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1325/2615] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:16.568042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:16.569759 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:16.591798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6aebd-6c50-4d52-bd07-7340e4b433e9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:16.600824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:16.601894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:16.602324 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:16.602815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] PATH_INFO: `/v3/projects/ff3541aae3f544c2927d89d1a1d9fe8f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:16.605606 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3541aae3f544c2927d89d1a1d9fe8f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:16.608056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:16.685045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3541aae3f544c2927d89d1a1d9fe8f', 'request_id': 'req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.693820 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3541aae3f544c2927d89d1a1d9fe8f', 'request_id': 'req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.701235 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3541aae3f544c2927d89d1a1d9fe8f', 'request_id': 'req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.719681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] The token cache is being invalidate because project ff3541aae3f544c2927d89d1a1d9fe8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:16.720684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.721437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b00334e7-5c1c-4cfb-b86d-d147ed6562d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:16.726415 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1291/2616] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:16 2022] DELETE /identity/v3/projects/ff3541aae3f544c2927d89d1a1d9fe8f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:16.728126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eca15-cd03-4ade-8f05-9fc9d52f491b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:16.728483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eca15-cd03-4ade-8f05-9fc9d52f491b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:16.728826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eca15-cd03-4ade-8f05-9fc9d52f491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:16.754019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c37eca15-cd03-4ade-8f05-9fc9d52f491b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:16.755239 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c37eca15-cd03-4ade-8f05-9fc9d52f491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:16.819717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1326/2617] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:16.829112 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e8a23-4e7a-416b-8e66-b26c847034ed None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:16.829951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e8a23-4e7a-416b-8e66-b26c847034ed None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:16.830228 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e8a23-4e7a-416b-8e66-b26c847034ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:16.852139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f37e8a23-4e7a-416b-8e66-b26c847034ed None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:16.853397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f37e8a23-4e7a-416b-8e66-b26c847034ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:16.866586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eca15-cd03-4ade-8f05-9fc9d52f491b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:16.873022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1292/2618] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:16.877367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eca15-cd03-4ade-8f05-9fc9d52f491b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:16.882117 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:16.882993 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:16.883585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:16.884220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:16.884793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:16.885160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:16.886863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:16.900551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1327/2619] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 15:04:16.904990 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e8a23-4e7a-416b-8e66-b26c847034ed None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:16.915769 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e8a23-4e7a-416b-8e66-b26c847034ed None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:16.929885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:16.929885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:16.931558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:16.931878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:16.933256 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:16.938055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:16.981906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1293/2620] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:16 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:16.986032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:16.986160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:16.988326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:17.057473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034fdde-be55-4949-ac21-88a5152b835a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:17.088318 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:17.107861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d58a5-0f9a-452e-9875-e590ff35796e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:17.118463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:17.119229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:17.119574 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:17.119919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:17.122712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:17.127363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:17.129546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:17.130810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:17.131237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:17.131764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:17.134962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:17.148385 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=82,keystone:DELETE=12,keystone:INSERT=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:17.157017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:17.159044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:17.173585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1328/2621] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:17.195261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1294/2622] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:16 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:17.196972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:17.209179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb7b98-2d14-4343-8226-3e0a13ad0f14 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:17.215124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:17.216796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:17.217327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:17.218286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:17.220176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:17.222151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:17.245100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1329/2623] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:17.245637 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:17.259779 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeafff-208b-4312-a9af-049739371dd6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:17.269497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01d2d4-0a24-4a96-b8b6-5b4e3c0e419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:17.270692 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d2d4-0a24-4a96-b8b6-5b4e3c0e419f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:17.271140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d2d4-0a24-4a96-b8b6-5b4e3c0e419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:17.271461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d2d4-0a24-4a96-b8b6-5b4e3c0e419f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:17.272508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01d2d4-0a24-4a96-b8b6-5b4e3c0e419f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:17.275683 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01d2d4-0a24-4a96-b8b6-5b4e3c0e419f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:17.294272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1295/2624] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:17.295809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:17.314342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c80b-d5da-47e1-95d9-5a62bd5225e3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:17.320827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:17.322010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:17.322291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:17.322560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] PATH_INFO: `/v3/projects/68e380562ff645bd9e14304dadcc21e9/users/87e1ff86f4a34baf9cf523e0f96d5fd4/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:17.341291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e380562ff645bd9e14304dadcc21e9, user_id=87e1ff86f4a34baf9cf523e0f96d5fd4, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:17.343953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:17.375938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1330/2625] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:17 2022] PUT /identity/v3/projects/68e380562ff645bd9e14304dadcc21e9/users/87e1ff86f4a34baf9cf523e0f96d5fd4/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:17.378211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a44b8-c2fc-4d14-8931-d9c63d2b7b86 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:17.378959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a44b8-c2fc-4d14-8931-d9c63d2b7b86 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:17.378959 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a44b8-c2fc-4d14-8931-d9c63d2b7b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:17.408772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2f9a44b8-c2fc-4d14-8931-d9c63d2b7b86 None None] MFA Rules not processed for user `87e1ff86f4a34baf9cf523e0f96d5fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:17.411355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2f9a44b8-c2fc-4d14-8931-d9c63d2b7b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:17.551410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1296/2626] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:17.564341 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:17.567614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:17.655396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b7a35-063b-466b-9aea-e2f32f7cccf9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:17.660797 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:17.661438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:17.661715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:17.662342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:17.671712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:17.677445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:17.691149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1331/2627] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:19.064608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a44b8-c2fc-4d14-8931-d9c63d2b7b86 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:19.410935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a44b8-c2fc-4d14-8931-d9c63d2b7b86 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:19.421165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:19.421845 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:19.422121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:19.422388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:19.426805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:19.559626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:19.562032 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:19.576250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1297/2628] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:21.823191 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:21.824845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:21.871764 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:21.873813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:21.925529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9f628-4c09-4961-a399-4a5d24b7cdee service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:21.930792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:21.931434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:21.931711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:21.931990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] PATH_INFO: `/v3/users/b72887db5ed34dbebe0d5745cf77c4a8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:21.933879 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72887db5ed34dbebe0d5745cf77c4a8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:21.936251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:21.947118 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:22.010265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bb7d0-3bc5-4ca3-8e6b-73a5112ce05c service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:22.018503 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:22.018811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.019512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.019898 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] PATH_INFO: `/v3/users/3c0c848f7858486c91773b34047d8aad` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.021735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0c848f7858486c91773b34047d8aad)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:22.025346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:22.026472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:22.052124 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72887db5ed34dbebe0d5745cf77c4a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.069001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72887db5ed34dbebe0d5745cf77c4a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.086991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72887db5ed34dbebe0d5745cf77c4a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.087905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72887db5ed34dbebe0d5745cf77c4a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.101468 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72887db5ed34dbebe0d5745cf77c4a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.112762 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1332/2629] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:21 2022] DELETE /identity/v3/users/b72887db5ed34dbebe0d5745cf77c4a8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:22.120299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0c848f7858486c91773b34047d8aad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.136956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0c848f7858486c91773b34047d8aad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.150887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0c848f7858486c91773b34047d8aad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.178526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0c848f7858486c91773b34047d8aad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.183709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0c848f7858486c91773b34047d8aad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.193678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1298/2630] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:21 2022] DELETE /identity/v3/users/3c0c848f7858486c91773b34047d8aad => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:22.395761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:22.397561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:22.461404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:22.463252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:22.539427 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853946f4-994a-41d0-bc7f-3ec3ea8aed3f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:22.541954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034465c8-16b5-4c35-baee-f0528d20c4c3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:22.551716 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:22.552360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.552621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.552888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] PATH_INFO: `/v3/projects/c83910e3ac2f4c8f9f127ab767bffd1a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.554058 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:22.554649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83910e3ac2f4c8f9f127ab767bffd1a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:22.555675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.555949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.556272 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] PATH_INFO: `/v3/projects/e498d32cc51a4bad9cb21e58985ce63e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.557394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:22.557641 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e498d32cc51a4bad9cb21e58985ce63e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:22.560720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:22.647335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83910e3ac2f4c8f9f127ab767bffd1a', 'request_id': 'req-1e07ac06-90b2-49de-b624-bbfe31c9decd'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.658045 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e498d32cc51a4bad9cb21e58985ce63e', 'request_id': 'req-863861b9-15c7-41d0-9848-3cb6e81a009e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.660781 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:22.661702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83910e3ac2f4c8f9f127ab767bffd1a', 'request_id': 'req-1e07ac06-90b2-49de-b624-bbfe31c9decd'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.668146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e498d32cc51a4bad9cb21e58985ce63e', 'request_id': 'req-863861b9-15c7-41d0-9848-3cb6e81a009e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.670977 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83910e3ac2f4c8f9f127ab767bffd1a', 'request_id': 'req-1e07ac06-90b2-49de-b624-bbfe31c9decd'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.678301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e498d32cc51a4bad9cb21e58985ce63e', 'request_id': 'req-863861b9-15c7-41d0-9848-3cb6e81a009e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.692810 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] The token cache is being invalidate because project c83910e3ac2f4c8f9f127ab767bffd1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:22.693723 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.694928 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1e07ac06-90b2-49de-b624-bbfe31c9decd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.696768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1333/2631] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:22 2022] DELETE /identity/v3/projects/c83910e3ac2f4c8f9f127ab767bffd1a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:22.702833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0ca9b-efda-4e79-8e0b-ca9ce9dc777e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.703185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0ca9b-efda-4e79-8e0b-ca9ce9dc777e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.703319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] The token cache is being invalidate because project e498d32cc51a4bad9cb21e58985ce63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:22.703517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0ca9b-efda-4e79-8e0b-ca9ce9dc777e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.703609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.704798 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-863861b9-15c7-41d0-9848-3cb6e81a009e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:22.706565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1299/2632] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:22 2022] DELETE /identity/v3/projects/e498d32cc51a4bad9cb21e58985ce63e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:22.710935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac3140-0337-4c1f-b14c-d7153e5dafdb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.711266 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac3140-0337-4c1f-b14c-d7153e5dafdb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.711484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac3140-0337-4c1f-b14c-d7153e5dafdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.731162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ccf0ca9b-efda-4e79-8e0b-ca9ce9dc777e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:22.732921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-aeac3140-0337-4c1f-b14c-d7153e5dafdb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:22.733054 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ccf0ca9b-efda-4e79-8e0b-ca9ce9dc777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:22.734199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-aeac3140-0337-4c1f-b14c-d7153e5dafdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:22.788887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:22.820860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1300/2633] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:22.823048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1334/2634] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:22.831438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09fd29-50f4-4aa2-93a8-8f5380371275 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.832043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8314-1996-480d-a273-6005d1825709 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.832043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8314-1996-480d-a273-6005d1825709 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.832216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8314-1996-480d-a273-6005d1825709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.832760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09fd29-50f4-4aa2-93a8-8f5380371275 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.833297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09fd29-50f4-4aa2-93a8-8f5380371275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.856962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-38ff8314-1996-480d-a273-6005d1825709 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:22.857419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ba09fd29-50f4-4aa2-93a8-8f5380371275 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:22.858658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ba09fd29-50f4-4aa2-93a8-8f5380371275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:22.858951 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-38ff8314-1996-480d-a273-6005d1825709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:22.870200 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1335/2635] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:22.870919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1301/2636] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:22.880381 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09fd29-50f4-4aa2-93a8-8f5380371275 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:22.884417 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8314-1996-480d-a273-6005d1825709 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:22.895928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8314-1996-480d-a273-6005d1825709 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:22.899066 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09fd29-50f4-4aa2-93a8-8f5380371275 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:22.901113 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:22.901948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.902227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.902502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.903155 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:22.903458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:22.905044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:22.907913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:22.909179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.909655 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.910359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.911321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:22.911782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:22.913586 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:22.927553 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1336/2637] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:22.927997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1302/2638] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:22.933699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:22.933858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:22.945966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ddfe3-77d7-4a20-92d5-5d7627545fa9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:22.946132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a26e1-e6c7-4b2f-bde1-74eb931bd3a9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:22.950945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:22.951633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.951984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.952331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.953085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:22.956249 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:22.957932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:22.958783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:22.959257 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:22.959722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:22.960808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:22.963202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:22.997651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1337/2639] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:22 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:23.003465 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.008221 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1303/2640] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:22 2022] POST /identity/v3/projects => generated 403 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:23.012163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.092294 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30deb36-8a21-48ed-96bc-cc29505340aa admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.098706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.098893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da5d65-9eb2-4a11-9a81-48a5abbfe73a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.099439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.099795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.100229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.100944 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.103093 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.103991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.104601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.104929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.105521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.106208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.108098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.133245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1338/2641] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:23 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:23.136543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.142914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1304/2642] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:23 2022] POST /identity/v3/users => generated 302 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:23.147814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.148651 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f17a7-c190-42ca-ad89-feae22ee8632 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.153930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.154534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.154815 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.155086 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.155725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.158511 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.169015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1339/2643] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:23.170423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69874fcc-db48-4cca-a7f3-2b470a11c43f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.172690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.175345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.175952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.176224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.176487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.177123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.178864 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.197680 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0331f0-7b23-4ced-ad1d-ba726ed1e3e8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.199004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1305/2644] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:23.199948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.215219 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.215833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.216108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.216377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.217020 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.220203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.226524 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341f2f6-3bf9-4e2f-94e7-7d7526f03a7d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.239327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1340/2645] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:23.243328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.244028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.244370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.244777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.245565 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.247349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.248823 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.273456 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1306/2646] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:23.277325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.280445 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918139e6-5253-497d-928c-ab4ed6d00229 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.286790 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:23.292201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840eff4-3da2-47f0-aa74-89555e1ad98f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.296088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d50bd-8fb1-4ac8-93f3-62782d88f156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.296886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d50bd-8fb1-4ac8-93f3-62782d88f156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.298286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d50bd-8fb1-4ac8-93f3-62782d88f156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.298575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d50bd-8fb1-4ac8-93f3-62782d88f156 admin admin] PATH_INFO: `/v3/projects/30651414d592424cb3836c8510d3a57e/users/5283ef11d0a949cc9b0c76c704e3a302/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.298784 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05b5f9-3c1f-46b7-8416-38f6708c305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.300249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b5f9-3c1f-46b7-8416-38f6708c305f admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.300383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b5f9-3c1f-46b7-8416-38f6708c305f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.300661 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b5f9-3c1f-46b7-8416-38f6708c305f admin admin] PATH_INFO: `/v3/projects/19b1902f648a4eeaa1c8ca8a667891eb/users/899e3717e5e6487f9ebe0fef487edb88/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.317782 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d50bd-8fb1-4ac8-93f3-62782d88f156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30651414d592424cb3836c8510d3a57e, user_id=5283ef11d0a949cc9b0c76c704e3a302, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.321509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d50bd-8fb1-4ac8-93f3-62782d88f156 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.323856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05b5f9-3c1f-46b7-8416-38f6708c305f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b1902f648a4eeaa1c8ca8a667891eb, user_id=899e3717e5e6487f9ebe0fef487edb88, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.328023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05b5f9-3c1f-46b7-8416-38f6708c305f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.345414 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1341/2647] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:23 2022] PUT /identity/v3/projects/30651414d592424cb3836c8510d3a57e/users/5283ef11d0a949cc9b0c76c704e3a302/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:23.357088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.357752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.358242 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.358657 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1307/2648] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:23 2022] PUT /identity/v3/projects/19b1902f648a4eeaa1c8ca8a667891eb/users/899e3717e5e6487f9ebe0fef487edb88/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:23.369925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2cdc-7503-4ac8-8b20-2e43cfddf955 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.370801 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2cdc-7503-4ac8-8b20-2e43cfddf955 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.371335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2cdc-7503-4ac8-8b20-2e43cfddf955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.412561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] MFA Rules not processed for user `5283ef11d0a949cc9b0c76c704e3a302`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:23.415498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:23.425013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f65a2cdc-7503-4ac8-8b20-2e43cfddf955 None None] MFA Rules not processed for user `899e3717e5e6487f9ebe0fef487edb88`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:23.426685 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f65a2cdc-7503-4ac8-8b20-2e43cfddf955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:23.466916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:23.510461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1342/2649] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:23.523896 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.526376 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:23.581356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1308/2650] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:23.586873 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2cdc-7503-4ac8-8b20-2e43cfddf955 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.665460 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3524c01-65e2-4969-aa07-e8c1a1e717ce None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.675923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.677105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.677646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.678375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.679146 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2cdc-7503-4ac8-8b20-2e43cfddf955 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.684232 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.684954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.685259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.685532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.686163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.688266 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.690155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.691792 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.703521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1343/2651] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:04:23.734618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1309/2652] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:23 2022] POST /identity/v3/projects => generated 405 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:23.740595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.761150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b9298-7589-42d0-b619-1223f6c4c733 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.857798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.859198 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.859543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.859874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.861293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.863812 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.905488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1344/2653] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:23 2022] POST /identity/v3/users => generated 309 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:23.910808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.923862 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e5fce-29c9-4c91-b02d-05ad19f08a09 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.930511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.932201 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.932698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.933183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:23.934280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:23.936264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:23.948981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1310/2654] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:23.953606 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:23.969448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54a1dd-865a-42a2-98a4-4f19ce5f6983 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:23.982119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:23.982824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:23.983195 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:23.983405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] PATH_INFO: `/v3/projects/13f3eeffa654431db2c5aacc94b1f775/users/cc776263e9df42ca8cb6d8f1857ed2a7/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:24.022598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f3eeffa654431db2c5aacc94b1f775, user_id=cc776263e9df42ca8cb6d8f1857ed2a7, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:24.029273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:24.046941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1345/2655] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:23 2022] PUT /identity/v3/projects/13f3eeffa654431db2c5aacc94b1f775/users/cc776263e9df42ca8cb6d8f1857ed2a7/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:24.051780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:24.062973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2b540-2a71-4733-b57f-6a01dece14be admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:24.129070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:24.166459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ef660-b0e1-4756-be3c-83a201b3a0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:24.167548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ef660-b0e1-4756-be3c-83a201b3a0fc admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:24.168215 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ef660-b0e1-4756-be3c-83a201b3a0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:24.168911 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ef660-b0e1-4756-be3c-83a201b3a0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:24.170025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ef660-b0e1-4756-be3c-83a201b3a0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:24.172672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ef660-b0e1-4756-be3c-83a201b3a0fc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:24.191773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1311/2656] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:24.193854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:24.210508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9fe37-2aa5-452f-8bf6-7ecac7b67021 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:24.216379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:24.217161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:24.217572 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:24.217959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] PATH_INFO: `/v3/projects/13f3eeffa654431db2c5aacc94b1f775/users/cc776263e9df42ca8cb6d8f1857ed2a7/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:24.221055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f3eeffa654431db2c5aacc94b1f775, user_id=cc776263e9df42ca8cb6d8f1857ed2a7, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:24.224250 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:24.247368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1346/2657] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:24 2022] PUT /identity/v3/projects/13f3eeffa654431db2c5aacc94b1f775/users/cc776263e9df42ca8cb6d8f1857ed2a7/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:24.250304 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb7eda-f4cf-4e37-86a1-78c15d5e94aa None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:24.250583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb7eda-f4cf-4e37-86a1-78c15d5e94aa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:24.250855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb7eda-f4cf-4e37-86a1-78c15d5e94aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:24.289783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b6eb7eda-f4cf-4e37-86a1-78c15d5e94aa None None] MFA Rules not processed for user `cc776263e9df42ca8cb6d8f1857ed2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:24.291159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b6eb7eda-f4cf-4e37-86a1-78c15d5e94aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:24.372234 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1312/2658] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:24.382555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:24.384394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:24.414071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:24.468420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7de84-9b04-488c-a943-656520d04f56 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:24.473958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df89eaf-cdd5-43d0-8f01-72c38da5bd20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:24.474493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df89eaf-cdd5-43d0-8f01-72c38da5bd20 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:24.474776 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df89eaf-cdd5-43d0-8f01-72c38da5bd20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:24.475040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df89eaf-cdd5-43d0-8f01-72c38da5bd20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:24.488877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df89eaf-cdd5-43d0-8f01-72c38da5bd20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:24.489302 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df89eaf-cdd5-43d0-8f01-72c38da5bd20 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:24.503307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1347/2659] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:24.832343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb7eda-f4cf-4e37-86a1-78c15d5e94aa None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:24.934389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb7eda-f4cf-4e37-86a1-78c15d5e94aa None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:24.941126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:24.942906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:24.942906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:24.942906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] PATH_INFO: `/v3/users/899e3717e5e6487f9ebe0fef487edb88` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:24.944015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=899e3717e5e6487f9ebe0fef487edb88)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:24.946063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:25.012322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899e3717e5e6487f9ebe0fef487edb88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.052900 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899e3717e5e6487f9ebe0fef487edb88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.055675 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:25.065184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899e3717e5e6487f9ebe0fef487edb88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.074659 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899e3717e5e6487f9ebe0fef487edb88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.075414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899e3717e5e6487f9ebe0fef487edb88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.081805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1313/2660] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:24 2022] DELETE /identity/v3/users/899e3717e5e6487f9ebe0fef487edb88 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:25.086557 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:25.087015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:25.087280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:25.107016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:25.108233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:25.171742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1348/2661] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:25.182137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:25.183693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:25.242913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b73520-9a2f-491f-9b7a-4a9a3cde59cc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:25.246828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:25.247550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:25.247831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:25.248098 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:25.258370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:25.259849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:25.269941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1314/2662] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:25.617290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:25.619005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:25.632430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc23d5-8551-404e-9319-2fad389451c7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:25.639534 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:25.640497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:25.640997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:25.642046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] PATH_INFO: `/v3/projects/19b1902f648a4eeaa1c8ca8a667891eb` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:25.644203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b1902f648a4eeaa1c8ca8a667891eb)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:25.646466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:25.717882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b1902f648a4eeaa1c8ca8a667891eb', 'request_id': 'req-304a1e04-e33b-4354-8779-60c30ea8ce40'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.725406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b1902f648a4eeaa1c8ca8a667891eb', 'request_id': 'req-304a1e04-e33b-4354-8779-60c30ea8ce40'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.731453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b1902f648a4eeaa1c8ca8a667891eb', 'request_id': 'req-304a1e04-e33b-4354-8779-60c30ea8ce40'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.744097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] The token cache is being invalidate because project 19b1902f648a4eeaa1c8ca8a667891eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:25.744475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.745166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.746793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1349/2663] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:25 2022] DELETE /identity/v3/projects/19b1902f648a4eeaa1c8ca8a667891eb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:25.750825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:25.763647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240a425-26f2-4add-88a7-9b749c348423 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:25.824249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:25.825320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:25.825823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:25.826356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] PATH_INFO: `/v3/users/cc776263e9df42ca8cb6d8f1857ed2a7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:25.827508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc776263e9df42ca8cb6d8f1857ed2a7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:25.830679 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:25.875985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc776263e9df42ca8cb6d8f1857ed2a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.887465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc776263e9df42ca8cb6d8f1857ed2a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.894680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc776263e9df42ca8cb6d8f1857ed2a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.901555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc776263e9df42ca8cb6d8f1857ed2a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.902161 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc776263e9df42ca8cb6d8f1857ed2a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:25.908755 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1315/2664] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:25 2022] DELETE /identity/v3/users/cc776263e9df42ca8cb6d8f1857ed2a7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:26.192746 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.194304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:26.218280 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1e04-e33b-4354-8779-60c30ea8ce40 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.264506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:26.288456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.289151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.289475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.289736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] PATH_INFO: `/v3/projects/13f3eeffa654431db2c5aacc94b1f775` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.291316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f3eeffa654431db2c5aacc94b1f775)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.293215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.325995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.327554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:26.339631 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:26.350074 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f3eeffa654431db2c5aacc94b1f775', 'request_id': 'req-d6774814-6f50-421a-9845-67d7a7b8c26a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:26.356539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f3eeffa654431db2c5aacc94b1f775', 'request_id': 'req-d6774814-6f50-421a-9845-67d7a7b8c26a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:26.363436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f3eeffa654431db2c5aacc94b1f775', 'request_id': 'req-d6774814-6f50-421a-9845-67d7a7b8c26a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:26.378092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] The token cache is being invalidate because project 13f3eeffa654431db2c5aacc94b1f775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:26.378385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:26.379005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6774814-6f50-421a-9845-67d7a7b8c26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:26.380492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1350/2665] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:26 2022] DELETE /identity/v3/projects/13f3eeffa654431db2c5aacc94b1f775 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:26.385161 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc334-c9df-47cf-bb91-e9c592efc68d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.385527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc334-c9df-47cf-bb91-e9c592efc68d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.385913 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc334-c9df-47cf-bb91-e9c592efc68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.392840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531b98f-59a5-4b94-9caa-e77a92c2771c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.405314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-397dc334-c9df-47cf-bb91-e9c592efc68d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:26.406777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-397dc334-c9df-47cf-bb91-e9c592efc68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:26.444126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.444743 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.445025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.445337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.447101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:26.466438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1351/2666] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:26.475510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f7900-cbf8-48d6-bc35-42e15a07b999 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.475791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f7900-cbf8-48d6-bc35-42e15a07b999 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.476064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f7900-cbf8-48d6-bc35-42e15a07b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.495405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c60f7900-cbf8-48d6-bc35-42e15a07b999 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:26.496590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c60f7900-cbf8-48d6-bc35-42e15a07b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:26.505215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1352/2667] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:26.509147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f7900-cbf8-48d6-bc35-42e15a07b999 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.511938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.513480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.517853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f7900-cbf8-48d6-bc35-42e15a07b999 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.522411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.522974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.523190 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1316/2668] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:04:26.523297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.523584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.524196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:26.524607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.526232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.537809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1353/2669] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:26.541991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.552010 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c61ece-2991-4981-ad6f-313e03e1b434 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.556699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.557338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.557620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.557887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.558512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.560441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.589817 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1317/2670] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:26 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:26.594672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.606091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7e694-e87c-4c7a-9285-4bafdd5c4ed1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.656359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:26.667991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.668608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.668883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.669153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.670930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.673448 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.695789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1354/2671] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:26 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:26.700141 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.710944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18968fce-8992-4905-8a6d-46b89f401d5e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.715797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.716409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.716684 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.716953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.717626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.719222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.731447 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1318/2672] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:26.736258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.747041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7f3d-7eab-4bb5-bf1d-1c34f066d929 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.751850 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d836f88-344a-4c50-9d9e-b65b1d052de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.752533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836f88-344a-4c50-9d9e-b65b1d052de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.752816 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836f88-344a-4c50-9d9e-b65b1d052de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.753011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836f88-344a-4c50-9d9e-b65b1d052de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.753725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d836f88-344a-4c50-9d9e-b65b1d052de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.755324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d836f88-344a-4c50-9d9e-b65b1d052de0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.767368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1355/2673] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:26.771674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.783834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9baeca-537c-42d0-bc1d-6cf8940dd6ba admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:26.791117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:26.792103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.792476 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.792991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] PATH_INFO: `/v3/projects/807f870a5e3f4a7ca628e5f267fb0213/users/5236fec08f3a4e329604d440445bbd97/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.811499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=807f870a5e3f4a7ca628e5f267fb0213, user_id=5236fec08f3a4e329604d440445bbd97, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:26.817815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:26.829129 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1319/2674] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:26 2022] PUT /identity/v3/projects/807f870a5e3f4a7ca628e5f267fb0213/users/5236fec08f3a4e329604d440445bbd97/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:26.839736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd6c8-a758-45dd-a14b-019130cd9b07 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:26.840049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd6c8-a758-45dd-a14b-019130cd9b07 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:26.840326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd6c8-a758-45dd-a14b-019130cd9b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:26.869985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5bacd6c8-a758-45dd-a14b-019130cd9b07 None None] MFA Rules not processed for user `5236fec08f3a4e329604d440445bbd97`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:26.871274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5bacd6c8-a758-45dd-a14b-019130cd9b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:26.937508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1356/2675] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:26.942074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:26.960075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:27.007224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ee6f6-d857-4a80-aa2a-63e0538c5b5b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:27.017388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:27.017388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.017678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.017946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.018836 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:27.020768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:27.051546 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1320/2676] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:26 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:27.055773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd6c8-a758-45dd-a14b-019130cd9b07 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:27.066344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd6c8-a758-45dd-a14b-019130cd9b07 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:27.123589 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:27.124286 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.124567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.124840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.125493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:27.127492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:27.149118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1357/2677] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:27 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:27.153181 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:27.164093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6cb79-1ef9-44f6-9e24-03ec5b314362 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:27.168717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:27.169334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.169616 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.169884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.170529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:27.172075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:27.182627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1321/2678] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:27.186853 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:27.196635 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623650c-6114-41f7-8fcc-583cfc6b86da admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:27.201331 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:27.201979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.202251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.202521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] PATH_INFO: `/v3/projects/a222e1acc2854686ae1f8b2cea450601/users/440ea34803384c58ae3275eed22ccb26/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.217714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a222e1acc2854686ae1f8b2cea450601, user_id=440ea34803384c58ae3275eed22ccb26, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:27.220923 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:27.236188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1358/2679] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:27 2022] PUT /identity/v3/projects/a222e1acc2854686ae1f8b2cea450601/users/440ea34803384c58ae3275eed22ccb26/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:27.240296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:27.250241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb310f-9355-433d-95da-de7f12624d2d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:27.306079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c093aa5-fabd-457a-a629-44a436c98c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:27.306596 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c093aa5-fabd-457a-a629-44a436c98c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.306869 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c093aa5-fabd-457a-a629-44a436c98c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.307137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c093aa5-fabd-457a-a629-44a436c98c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.307772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c093aa5-fabd-457a-a629-44a436c98c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:27.309366 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c093aa5-fabd-457a-a629-44a436c98c57 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:27.319055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1322/2680] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:27.323284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:27.334293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345abe-ad94-4d9e-bab3-d67d30360328 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:27.339040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:27.339681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.339985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.340266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] PATH_INFO: `/v3/projects/a222e1acc2854686ae1f8b2cea450601/users/440ea34803384c58ae3275eed22ccb26/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.342182 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a222e1acc2854686ae1f8b2cea450601, user_id=440ea34803384c58ae3275eed22ccb26, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:27.345425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:27.355937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1359/2681] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:27 2022] PUT /identity/v3/projects/a222e1acc2854686ae1f8b2cea450601/users/440ea34803384c58ae3275eed22ccb26/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:27.367174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c242d-16e1-4382-934e-e7bcd58a2274 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.367682 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c242d-16e1-4382-934e-e7bcd58a2274 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.367780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c242d-16e1-4382-934e-e7bcd58a2274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.396338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-724c242d-16e1-4382-934e-e7bcd58a2274 None None] MFA Rules not processed for user `440ea34803384c58ae3275eed22ccb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:27.397627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-724c242d-16e1-4382-934e-e7bcd58a2274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:27.472297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1323/2682] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:27.483529 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:27.485327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:27.499361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:27.545017 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3a36c-0809-47db-903b-3c768a085911 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:27.549273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e46c92-be13-4446-81c1-db4f01a8a840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:27.549978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46c92-be13-4446-81c1-db4f01a8a840 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:27.550325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46c92-be13-4446-81c1-db4f01a8a840 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:27.550630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46c92-be13-4446-81c1-db4f01a8a840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:27.561347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e46c92-be13-4446-81c1-db4f01a8a840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:27.562856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e46c92-be13-4446-81c1-db4f01a8a840 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:27.575802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1360/2683] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:28.071011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c242d-16e1-4382-934e-e7bcd58a2274 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:28.083410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:28.136578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c242d-16e1-4382-934e-e7bcd58a2274 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:28.141028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:28.141700 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:28.141994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:28.142259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] PATH_INFO: `/v3/users/5236fec08f3a4e329604d440445bbd97` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:28.143404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5236fec08f3a4e329604d440445bbd97)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:28.145283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:28.202888 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5236fec08f3a4e329604d440445bbd97'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:28.217242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5236fec08f3a4e329604d440445bbd97'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:28.225326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5236fec08f3a4e329604d440445bbd97'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:28.235532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5236fec08f3a4e329604d440445bbd97'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:28.236853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5236fec08f3a4e329604d440445bbd97'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:28.250431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1324/2684] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:28 2022] DELETE /identity/v3/users/5236fec08f3a4e329604d440445bbd97 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:28.254892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:28.255980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:28.256525 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:28.276961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:28.278795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:28.348100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1361/2685] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:28.357976 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:28.361948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:28.468584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d26566-0a22-40a6-900b-08f34dadb440 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:28.473393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:28.474023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:28.474298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:28.474566 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:28.485824 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:28.487269 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:28.499669 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1325/2686] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:04:28.836475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:28.838839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:28.852266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76b61f-6de8-464b-bc73-fae08e8f5d38 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:28.857872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:28.858768 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:28.859112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:28.859415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] PATH_INFO: `/v3/projects/807f870a5e3f4a7ca628e5f267fb0213` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:28.861053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=807f870a5e3f4a7ca628e5f267fb0213)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:28.863138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:28.967369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807f870a5e3f4a7ca628e5f267fb0213', 'request_id': 'req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:28.977583 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807f870a5e3f4a7ca628e5f267fb0213', 'request_id': 'req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:28.986600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807f870a5e3f4a7ca628e5f267fb0213', 'request_id': 'req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.005774 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] The token cache is being invalidate because project 807f870a5e3f4a7ca628e5f267fb0213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:29.006237 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.007224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.008885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1362/2687] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:28 2022] DELETE /identity/v3/projects/807f870a5e3f4a7ca628e5f267fb0213 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:29.013039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:29.025795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c9f1d-7c5e-4d24-9a5f-8ca552804aa5 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:29.077813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:29.112410 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:29.113046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:29.113369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:29.113647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] PATH_INFO: `/v3/users/440ea34803384c58ae3275eed22ccb26` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:29.114814 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=440ea34803384c58ae3275eed22ccb26)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:29.116730 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:29.171332 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440ea34803384c58ae3275eed22ccb26'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.184328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440ea34803384c58ae3275eed22ccb26'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.192167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440ea34803384c58ae3275eed22ccb26'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.203003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440ea34803384c58ae3275eed22ccb26'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.203899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c9926505-ad08-4170-8133-026828c0bd87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440ea34803384c58ae3275eed22ccb26'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.213287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1326/2688] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:29 2022] DELETE /identity/v3/users/440ea34803384c58ae3275eed22ccb26 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:29.528717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:29.530196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:29.553327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfa47e-7243-4a9b-b6cd-9d833fff8901 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:29.589145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:29.618765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:29.619579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:29.619954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:29.620325 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] PATH_INFO: `/v3/projects/a222e1acc2854686ae1f8b2cea450601` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:29.621679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a222e1acc2854686ae1f8b2cea450601)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:29.624008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:29.687794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a222e1acc2854686ae1f8b2cea450601', 'request_id': 'req-add2f635-fafe-4139-9283-457c4e8ea8ab'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.696136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a222e1acc2854686ae1f8b2cea450601', 'request_id': 'req-add2f635-fafe-4139-9283-457c4e8ea8ab'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.706469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a222e1acc2854686ae1f8b2cea450601', 'request_id': 'req-add2f635-fafe-4139-9283-457c4e8ea8ab'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.725444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] The token cache is being invalidate because project a222e1acc2854686ae1f8b2cea450601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:29.725741 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.726617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-add2f635-fafe-4139-9283-457c4e8ea8ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:29.728422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1363/2689] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:29 2022] DELETE /identity/v3/projects/a222e1acc2854686ae1f8b2cea450601 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:29.733174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d85d70-ba58-4913-9880-8bdfbe74e673 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:29.733686 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d85d70-ba58-4913-9880-8bdfbe74e673 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:29.733775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d85d70-ba58-4913-9880-8bdfbe74e673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:29.754608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c6d85d70-ba58-4913-9880-8bdfbe74e673 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:29.756199 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c6d85d70-ba58-4913-9880-8bdfbe74e673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:29.856418 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1327/2690] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:29.864560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169482b-262f-408e-bd50-94ec311cc610 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:29.864846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169482b-262f-408e-bd50-94ec311cc610 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:29.865162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169482b-262f-408e-bd50-94ec311cc610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:29.885795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5169482b-262f-408e-bd50-94ec311cc610 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:29.889924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5169482b-262f-408e-bd50-94ec311cc610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:29.903631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1364/2691] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:29.904843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d85d70-ba58-4913-9880-8bdfbe74e673 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:29.915735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d85d70-ba58-4913-9880-8bdfbe74e673 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:29.920623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:29.921250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:29.921831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:29.922153 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:29.922791 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:29.923761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:29.924835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:29.941368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1328/2692] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:29.943614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169482b-262f-408e-bd50-94ec311cc610 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:29.956715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169482b-262f-408e-bd50-94ec311cc610 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:29.963764 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:29.964569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:29.964908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:29.965507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:29.966254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:29.968388 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.006430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1365/2693] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:29 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:30.020302 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.033319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1def48-6e84-4782-be24-90082000e1f1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.077928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:30.102962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.104074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.104074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.104349 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.105088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.107159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.141955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1329/2694] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:30 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:30.148083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.167678 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340001a-36c6-4691-8dd9-02c2cbb80169 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.174040 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.174753 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.174956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.175221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.175877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.178343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.194544 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1366/2695] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:30.199323 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.211696 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6764b75-d327-4432-aaa7-9ac510d492cc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.217027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dfa19-c6b0-4454-88be-d56795b74897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.217687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dfa19-c6b0-4454-88be-d56795b74897 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.218072 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dfa19-c6b0-4454-88be-d56795b74897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.218423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dfa19-c6b0-4454-88be-d56795b74897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.219132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dfa19-c6b0-4454-88be-d56795b74897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.221041 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dfa19-c6b0-4454-88be-d56795b74897 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.232487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1330/2696] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:30.238041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.252690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72230fac-c126-4f55-b57a-cd5ea593db31 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.258340 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.259064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.259497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.259806 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] PATH_INFO: `/v3/projects/41ed85840b3646a4936e7b3d92c6946c/users/6d05caddddc54e999c0c0ec906bc8d57/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.276849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ed85840b3646a4936e7b3d92c6946c, user_id=6d05caddddc54e999c0c0ec906bc8d57, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.279813 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.301297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1367/2697] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:30 2022] PUT /identity/v3/projects/41ed85840b3646a4936e7b3d92c6946c/users/6d05caddddc54e999c0c0ec906bc8d57/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:30.308711 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d00c4-cc2d-40a9-bfc2-1dd613fc2f91 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.309062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d00c4-cc2d-40a9-bfc2-1dd613fc2f91 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.309469 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d00c4-cc2d-40a9-bfc2-1dd613fc2f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.342948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-976d00c4-cc2d-40a9-bfc2-1dd613fc2f91 None None] MFA Rules not processed for user `6d05caddddc54e999c0c0ec906bc8d57`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:30.344390 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-976d00c4-cc2d-40a9-bfc2-1dd613fc2f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:30.448028 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1331/2698] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:30.452509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.524455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db09a93-cadd-4af6-8fe7-b42879a29b65 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.529467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.530059 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.530338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.530601 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.531692 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.534165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.544897 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:30.569230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1368/2699] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:30 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:30.571200 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d00c4-cc2d-40a9-bfc2-1dd613fc2f91 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.582271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d00c4-cc2d-40a9-bfc2-1dd613fc2f91 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.644397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.644922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.645222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.645495 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.646139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.648076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.669758 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1332/2700] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:30 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:30.674174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.684487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4816aa-4ff1-4f5d-8edf-113b209e9958 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.689303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.689972 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.690319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.690672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.691374 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.693045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.703798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1369/2701] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:30.707795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.719435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f087b-f3b9-4508-898e-6f641fbb332a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.724688 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.725368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.725649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.725921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] PATH_INFO: `/v3/projects/cc100efb2e26474aa6590c71abddcc40/users/e83a87dbc035481aa7681f53416a7e86/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.741547 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc100efb2e26474aa6590c71abddcc40, user_id=e83a87dbc035481aa7681f53416a7e86, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.744191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.757128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1333/2702] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:30 2022] PUT /identity/v3/projects/cc100efb2e26474aa6590c71abddcc40/users/e83a87dbc035481aa7681f53416a7e86/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:30.761119 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.771232 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa5864-8960-453a-8072-e5684415c7ff admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.826899 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ce83a-1f62-4ea9-91b6-5ae7361cb7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.827503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ce83a-1f62-4ea9-91b6-5ae7361cb7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.827778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ce83a-1f62-4ea9-91b6-5ae7361cb7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.828042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ce83a-1f62-4ea9-91b6-5ae7361cb7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.828674 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ce83a-1f62-4ea9-91b6-5ae7361cb7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.830796 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ce83a-1f62-4ea9-91b6-5ae7361cb7f9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.840864 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1370/2703] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:30.846064 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:30.856793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fc467-8af9-44c9-9827-416f5a777276 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:30.861525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:30.862273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.862617 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.862953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] PATH_INFO: `/v3/projects/cc100efb2e26474aa6590c71abddcc40/users/e83a87dbc035481aa7681f53416a7e86/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.864967 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc100efb2e26474aa6590c71abddcc40, user_id=e83a87dbc035481aa7681f53416a7e86, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:30.868823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:30.877722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:30.883109 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1334/2704] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:30 2022] PUT /identity/v3/projects/cc100efb2e26474aa6590c71abddcc40/users/e83a87dbc035481aa7681f53416a7e86/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:30.895564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3168e-2d21-4ce9-b505-93989288566d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:30.895841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3168e-2d21-4ce9-b505-93989288566d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:30.896158 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3168e-2d21-4ce9-b505-93989288566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:30.927068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-21d3168e-2d21-4ce9-b505-93989288566d None None] MFA Rules not processed for user `e83a87dbc035481aa7681f53416a7e86`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:30.928319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-21d3168e-2d21-4ce9-b505-93989288566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:30.999356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1371/2705] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:31.013695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:31.013695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:31.071620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960fa16-31a8-4d5e-9952-a36877b35fed admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:31.075276 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:31.075887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.076167 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.076430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.086488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:31.087893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:31.098960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1335/2706] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:31.187737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3168e-2d21-4ce9-b505-93989288566d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:31.198719 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3168e-2d21-4ce9-b505-93989288566d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:31.202711 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:31.203322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.203631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.203903 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.271683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:31.273301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:31.277267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:31.285419 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1372/2707] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:31.303887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:31.305411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:31.403706 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5509a-0a53-40be-9590-9aa9f2ae9888 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:31.411464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:31.412357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.412960 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.413267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] PATH_INFO: `/v3/users/87e1ff86f4a34baf9cf523e0f96d5fd4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.414623 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e1ff86f4a34baf9cf523e0f96d5fd4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:31.416687 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:31.511262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e1ff86f4a34baf9cf523e0f96d5fd4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.523791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:31.525296 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:31.539341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e1ff86f4a34baf9cf523e0f96d5fd4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.547252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ae162-6fb4-4847-a934-81a25cb498df service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:31.553961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:31.554621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.554855 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e1ff86f4a34baf9cf523e0f96d5fd4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.555031 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.555175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] PATH_INFO: `/v3/users/5283ef11d0a949cc9b0c76c704e3a302` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.556475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5283ef11d0a949cc9b0c76c704e3a302)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:31.558645 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:31.567501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e1ff86f4a34baf9cf523e0f96d5fd4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.568222 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3311bc78-4ef9-4700-ab32-828dc88b37c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e1ff86f4a34baf9cf523e0f96d5fd4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.577023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1336/2708] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:31 2022] DELETE /identity/v3/users/87e1ff86f4a34baf9cf523e0f96d5fd4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:31.581519 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.581916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.582288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.607341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:31.608989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:31.615822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5283ef11d0a949cc9b0c76c704e3a302'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.623029 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5283ef11d0a949cc9b0c76c704e3a302'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.632529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:31.637035 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5283ef11d0a949cc9b0c76c704e3a302'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.638006 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5283ef11d0a949cc9b0c76c704e3a302'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.646492 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-389fdbd5-d776-4041-a873-23ce98dadb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5283ef11d0a949cc9b0c76c704e3a302'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:31.653803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1373/2709] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:31 2022] DELETE /identity/v3/users/5283ef11d0a949cc9b0c76c704e3a302 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:31.658359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.658857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.659133 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.680316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:31.681564 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:31.690149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1337/2710] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:31.700494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:31.702393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:31.749915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1374/2711] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:31.759934 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:31.762426 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:31.776520 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fe8d9-4417-4b95-8b4f-dbb80df1d96d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:31.840158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8802c83-1482-44a0-a7aa-58de08e085ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:31.841490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8802c83-1482-44a0-a7aa-58de08e085ad service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.841909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8802c83-1482-44a0-a7aa-58de08e085ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.842284 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8802c83-1482-44a0-a7aa-58de08e085ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.842410 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833003d-2602-4e0b-8462-c6ae85643f4d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:31.847269 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:31.847943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:31.848363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:31.848695 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:31.855647 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8802c83-1482-44a0-a7aa-58de08e085ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:31.857212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8802c83-1482-44a0-a7aa-58de08e085ad service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:31.863726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:31.865273 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:31.876737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1375/2712] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:04:31.877401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1338/2713] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:04:32.296220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:32.298279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:32.317545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08714214-358f-408d-a923-16e0e1ad7c24 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:32.322643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:32.323386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.324024 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.324811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] PATH_INFO: `/v3/projects/68e380562ff645bd9e14304dadcc21e9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.326108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e380562ff645bd9e14304dadcc21e9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:32.328107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:32.344521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:32.399979 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e380562ff645bd9e14304dadcc21e9', 'request_id': 'req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.407688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e380562ff645bd9e14304dadcc21e9', 'request_id': 'req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.414370 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e380562ff645bd9e14304dadcc21e9', 'request_id': 'req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.429927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] The token cache is being invalidate because project 68e380562ff645bd9e14304dadcc21e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:32.430220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.430933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.433251 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1376/2714] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:32 2022] DELETE /identity/v3/projects/68e380562ff645bd9e14304dadcc21e9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:32.437421 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891e22c-4784-4808-8c63-e0ce9dfcfea7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.437733 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891e22c-4784-4808-8c63-e0ce9dfcfea7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.438006 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891e22c-4784-4808-8c63-e0ce9dfcfea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.459773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6891e22c-4784-4808-8c63-e0ce9dfcfea7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:32.459773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6891e22c-4784-4808-8c63-e0ce9dfcfea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:32.461874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:32.463556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:32.549290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1339/2715] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:32.552538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482ccb-59a5-4d04-89f2-f3ca65cbc98d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.552840 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482ccb-59a5-4d04-89f2-f3ca65cbc98d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.553119 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482ccb-59a5-4d04-89f2-f3ca65cbc98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.562927 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2185e2b-1e46-4bfe-9bac-f4ae279693c9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:32.578785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:32.580039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f3482ccb-59a5-4d04-89f2-f3ca65cbc98d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:32.580209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.580585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.580867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] PATH_INFO: `/v3/projects/30651414d592424cb3836c8510d3a57e` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.581259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f3482ccb-59a5-4d04-89f2-f3ca65cbc98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:32.583439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30651414d592424cb3836c8510d3a57e)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:32.585975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:32.591139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1340/2716] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:32.595955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3482ccb-59a5-4d04-89f2-f3ca65cbc98d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:32.611020 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3482ccb-59a5-4d04-89f2-f3ca65cbc98d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:32.621213 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:32.621918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.622266 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.622654 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.623506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:32.623909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:32.625965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:32.642909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1341/2717] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:32.649679 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:32.659085 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30651414d592424cb3836c8510d3a57e', 'request_id': 'req-5c6d4f65-218f-4928-83b5-3db5ab751103'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.663539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:32.667022 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30651414d592424cb3836c8510d3a57e', 'request_id': 'req-5c6d4f65-218f-4928-83b5-3db5ab751103'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.671904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e23ac2-49a1-451f-be05-0b19c393b1c6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:32.674222 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30651414d592424cb3836c8510d3a57e', 'request_id': 'req-5c6d4f65-218f-4928-83b5-3db5ab751103'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.698904 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] The token cache is being invalidate because project 30651414d592424cb3836c8510d3a57e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:32.699278 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.700196 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5c6d4f65-218f-4928-83b5-3db5ab751103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:32.708226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1377/2718] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:32 2022] DELETE /identity/v3/projects/30651414d592424cb3836c8510d3a57e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:32.715666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650245b-6282-4b29-acda-80f4394ffb97 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.715666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650245b-6282-4b29-acda-80f4394ffb97 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.715666 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650245b-6282-4b29-acda-80f4394ffb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.756631 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0650245b-6282-4b29-acda-80f4394ffb97 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:32.758499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0650245b-6282-4b29-acda-80f4394ffb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:32.815629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:32.816738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:32.817431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.818298 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.818461 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.819322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:32.822691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:32.883709 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1378/2719] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:32.895233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646595c2-350d-4e64-97e4-b000612165f9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:32.895960 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646595c2-350d-4e64-97e4-b000612165f9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:32.896926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646595c2-350d-4e64-97e4-b000612165f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:32.901280 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1342/2720] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:32 2022] POST /identity/v3/projects => generated 381 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:32.902795 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:32.906490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:32.933496 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-646595c2-350d-4e64-97e4-b000612165f9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:32.935671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-646595c2-350d-4e64-97e4-b000612165f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:33.004880 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a35638-8845-41c1-b6d5-390be3c2f58c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.011433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.012158 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.012437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.012710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.013393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.017337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.028706 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1379/2721] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:33.033998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646595c2-350d-4e64-97e4-b000612165f9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.045785 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646595c2-350d-4e64-97e4-b000612165f9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.053652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1343/2722] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:32 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:33.054245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.054534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.055463 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.055852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.056673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.059007 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:33.059628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.062443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.064871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb8884-3e80-4b78-b616-7f250e25c8df admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.069978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.070587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.070870 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.071143 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.071789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.073411 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.084943 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1344/2723] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:33.088996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.096800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1380/2724] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 15:04:33.098058 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.105526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8f6f-49ad-466e-9bca-ddd08763859f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.109228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625f691-f736-49cf-b00c-a556441b335c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.112770 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.113416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.113751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.114029 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.114670 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.115442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.116268 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.116383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.116865 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.117385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.118282 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.120436 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.127750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1345/2725] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:33.132005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.142726 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c7200-9347-4914-a6e3-b8914cef3214 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.148944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736cb97-5a6e-4fcd-b9f2-1fa19ff9cd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.150091 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736cb97-5a6e-4fcd-b9f2-1fa19ff9cd06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.150374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736cb97-5a6e-4fcd-b9f2-1fa19ff9cd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.150710 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736cb97-5a6e-4fcd-b9f2-1fa19ff9cd06 admin admin] PATH_INFO: `/v3/projects/781a403f041b4389b804107c30ffedac/users/33a5368bd6fb4ebeba76c5f6ce0c7c28/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.168487 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1381/2726] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:33 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:33.170206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.174026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736cb97-5a6e-4fcd-b9f2-1fa19ff9cd06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781a403f041b4389b804107c30ffedac, user_id=33a5368bd6fb4ebeba76c5f6ce0c7c28, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.176856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736cb97-5a6e-4fcd-b9f2-1fa19ff9cd06 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.195290 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1346/2727] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:33 2022] PUT /identity/v3/projects/781a403f041b4389b804107c30ffedac/users/33a5368bd6fb4ebeba76c5f6ce0c7c28/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:33.201977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.202497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.202922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.261103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:33.265000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:33.304173 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3736b-d9fc-4c26-8c41-0a1722ba51f1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.307909 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:33.356178 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1347/2728] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:33.366731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:33.374916 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.377316 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:33.395736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.396543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.396855 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.397256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.397965 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.400171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.430550 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1382/2729] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:33 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:33.435996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.449324 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285eeda-82bf-4c25-bd10-ca517acdb5ae admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.468573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.469224 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.470028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.470308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.471046 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.473431 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.487980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1383/2730] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:33.491004 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74224d4f-6d8c-4fe6-90e9-ccf1e8a19e55 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.491627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.495451 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.496147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.496932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.497312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.502371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26c09d-4513-4915-9191-db525123d00c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.507336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37db1c-b0c6-46d1-a8fa-01423d48137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.508466 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37db1c-b0c6-46d1-a8fa-01423d48137d admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.508754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37db1c-b0c6-46d1-a8fa-01423d48137d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.509160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37db1c-b0c6-46d1-a8fa-01423d48137d admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.509995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37db1c-b0c6-46d1-a8fa-01423d48137d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.510883 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.511730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37db1c-b0c6-46d1-a8fa-01423d48137d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.512807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.527187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1348/2731] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:04:33.530727 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.540683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1384/2732] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:33.549252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df272c9-f35c-41a0-8d10-a6d849ba3062 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.556498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.557259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.557564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.557832 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] PATH_INFO: `/v3/projects/1d18f2167d98453e9d8d56619b7e53b7/users/ea20af879c5647208871f2114e1b77ee/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.583334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d18f2167d98453e9d8d56619b7e53b7, user_id=ea20af879c5647208871f2114e1b77ee, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:33.587121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:33.610171 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1349/2733] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:33 2022] PUT /identity/v3/projects/1d18f2167d98453e9d8d56619b7e53b7/users/ea20af879c5647208871f2114e1b77ee/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:33.622438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.623647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.624422 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.679628 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] MFA Rules not processed for user `ea20af879c5647208871f2114e1b77ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:33.680895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:33.819039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1385/2734] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:33.831301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.833634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:33.967913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d4324-ef01-4b08-81dc-fc725abd25bd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:33.976021 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c247a4-ef52-4fbc-8b2b-b797a428b3f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:33.980163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c247a4-ef52-4fbc-8b2b-b797a428b3f1 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:33.980163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c247a4-ef52-4fbc-8b2b-b797a428b3f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:33.980397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c247a4-ef52-4fbc-8b2b-b797a428b3f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:33.991194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:33.991194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.004921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c247a4-ef52-4fbc-8b2b-b797a428b3f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:34.007721 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c247a4-ef52-4fbc-8b2b-b797a428b3f1 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:34.025912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1350/2735] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:34.084841 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09808a-b28a-45ce-8a0d-f750981e6b1f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:34.097457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:34.098390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:34.098737 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:34.099082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] PATH_INFO: `/v3/users/0eccc82843084cb392b1f15ad2ac9268` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:34.100973 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eccc82843084cb392b1f15ad2ac9268)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:34.104079 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:34.173705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:34.180955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eccc82843084cb392b1f15ad2ac9268'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.191144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eccc82843084cb392b1f15ad2ac9268'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.208881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eccc82843084cb392b1f15ad2ac9268'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.209766 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eccc82843084cb392b1f15ad2ac9268'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.218866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eccc82843084cb392b1f15ad2ac9268'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.227824 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1386/2736] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:33 2022] DELETE /identity/v3/users/0eccc82843084cb392b1f15ad2ac9268 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:34.358058 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:34.358249 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:34.358591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:34.393346 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] MFA Rules not processed for user `33a5368bd6fb4ebeba76c5f6ce0c7c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:34.394896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.435648 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:34.437136 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.458540 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:34.549941 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1351/2737] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:34.550928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:34.552268 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.619288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b0baa-8668-46d2-bab6-b8f971f2dabd admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:34.634889 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:34.635546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:34.635827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:34.636092 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] PATH_INFO: `/v3/users/6d05caddddc54e999c0c0ec906bc8d57` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:34.637622 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d05caddddc54e999c0c0ec906bc8d57)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:34.640472 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:34.666023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db9ead-19ba-408e-a745-7ee91927c80e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:34.674056 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:34.674965 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:34.675267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:34.675590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] PATH_INFO: `/v3/projects/3f711f25c212427db970f02391c79d1c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:34.676958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f711f25c212427db970f02391c79d1c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:34.678936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:34.705112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d05caddddc54e999c0c0ec906bc8d57'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.715602 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d05caddddc54e999c0c0ec906bc8d57'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.729271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d05caddddc54e999c0c0ec906bc8d57'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.730575 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d05caddddc54e999c0c0ec906bc8d57'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.740524 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d05caddddc54e999c0c0ec906bc8d57'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.742340 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f711f25c212427db970f02391c79d1c', 'request_id': 'req-8df72ee0-9dc9-445a-b704-6dba1e16825e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.749749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1387/2738] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:34 2022] DELETE /identity/v3/users/6d05caddddc54e999c0c0ec906bc8d57 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:34.750389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f711f25c212427db970f02391c79d1c', 'request_id': 'req-8df72ee0-9dc9-445a-b704-6dba1e16825e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.754172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:34.755793 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.758402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f711f25c212427db970f02391c79d1c', 'request_id': 'req-8df72ee0-9dc9-445a-b704-6dba1e16825e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.772757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] The token cache is being invalidate because project 3f711f25c212427db970f02391c79d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:34.773055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.773802 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8df72ee0-9dc9-445a-b704-6dba1e16825e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.775334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1352/2739] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:34 2022] DELETE /identity/v3/projects/3f711f25c212427db970f02391c79d1c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:34.780154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:34.780154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:34.780154 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:34.800489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:34.802156 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.809052 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:34.833759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12702d72-170e-47b2-bad2-6e226ce6bec7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:34.837952 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:34.838629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:34.838970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:34.839336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:34.841162 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.859876 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:34.874471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1353/2740] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:34.875564 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:34.877580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.893094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decb389-3519-4bf8-bada-afddf19050e9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:34.899182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:34.899809 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:34.900400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:34.900699 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] PATH_INFO: `/v3/users/82bda11d02fd4147a24800327159c539` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:34.901846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bda11d02fd4147a24800327159c539)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:34.904024 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:34.916938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:34.918404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:34.928002 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1388/2741] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:34.928784 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:34.930316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:34.957000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bda11d02fd4147a24800327159c539'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.970474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bda11d02fd4147a24800327159c539'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.978592 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bda11d02fd4147a24800327159c539'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.987502 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bda11d02fd4147a24800327159c539'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.988319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bda11d02fd4147a24800327159c539'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:34.994147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e2ede-b79d-46dd-b7bc-05474d2a321f service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:34.995241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1354/2742] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:34 2022] DELETE /identity/v3/users/82bda11d02fd4147a24800327159c539 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:35.069821 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:35.070424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:35.070697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:35.070963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:35.072885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:35.149560 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:35.155308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:35.158322 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:35.171034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1389/2743] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:35.188866 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:35.190446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:35.206448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e7643-78d0-49f2-b2b4-4316788bd768 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:35.212296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:35.213650 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:35.214022 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:35.214515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] PATH_INFO: `/v3/projects/6ea109160a154ce6a71660bdcf306133` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:35.237843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea109160a154ce6a71660bdcf306133)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:35.242578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:35.309828 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea109160a154ce6a71660bdcf306133', 'request_id': 'req-248763d6-c490-4a39-91d6-ec3b72497a06'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.317472 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea109160a154ce6a71660bdcf306133', 'request_id': 'req-248763d6-c490-4a39-91d6-ec3b72497a06'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.324829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea109160a154ce6a71660bdcf306133', 'request_id': 'req-248763d6-c490-4a39-91d6-ec3b72497a06'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.341924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] The token cache is being invalidate because project 6ea109160a154ce6a71660bdcf306133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:35.342216 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.342914 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.345604 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1355/2744] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:35 2022] DELETE /identity/v3/projects/6ea109160a154ce6a71660bdcf306133 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:35.507621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:35.509089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:35.522587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e802c5c-646d-458b-b3bb-59b89fda3c09 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:35.598106 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:35.599009 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:35.599291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:35.599763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] PATH_INFO: `/v3/projects/41ed85840b3646a4936e7b3d92c6946c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:35.601603 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ed85840b3646a4936e7b3d92c6946c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:35.604765 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:35.655980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ed85840b3646a4936e7b3d92c6946c', 'request_id': 'req-788c767b-b57b-4a95-a288-d1ec6451d0a1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.663307 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ed85840b3646a4936e7b3d92c6946c', 'request_id': 'req-788c767b-b57b-4a95-a288-d1ec6451d0a1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.668900 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ed85840b3646a4936e7b3d92c6946c', 'request_id': 'req-788c767b-b57b-4a95-a288-d1ec6451d0a1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.681807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] The token cache is being invalidate because project 41ed85840b3646a4936e7b3d92c6946c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:35.682087 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.682744 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.684221 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1390/2745] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:35 2022] DELETE /identity/v3/projects/41ed85840b3646a4936e7b3d92c6946c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:35.688822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:35.690252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:35.703423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248763d6-c490-4a39-91d6-ec3b72497a06 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:35.756176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:35.768169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:35.769003 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:35.769667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:35.770100 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] PATH_INFO: `/v3/users/e83a87dbc035481aa7681f53416a7e86` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:35.771433 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e83a87dbc035481aa7681f53416a7e86)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:35.773402 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:35.821966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e83a87dbc035481aa7681f53416a7e86'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.835165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e83a87dbc035481aa7681f53416a7e86'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.842187 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e83a87dbc035481aa7681f53416a7e86'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.851484 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e83a87dbc035481aa7681f53416a7e86'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.852172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-af91503a-873a-4255-8a73-d45e01b05be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e83a87dbc035481aa7681f53416a7e86'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:35.858713 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1356/2746] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:35 2022] DELETE /identity/v3/users/e83a87dbc035481aa7681f53416a7e86 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:36.363801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.365408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:36.381535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e944302-927d-4aee-9ab9-5ba62f0cb778 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.381933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e944302-927d-4aee-9ab9-5ba62f0cb778 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.382629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e944302-927d-4aee-9ab9-5ba62f0cb778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.389610 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c767b-b57b-4a95-a288-d1ec6451d0a1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.403334 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1e944302-927d-4aee-9ab9-5ba62f0cb778 None None] MFA Rules not processed for user `33a5368bd6fb4ebeba76c5f6ce0c7c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:36.404611 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1e944302-927d-4aee-9ab9-5ba62f0cb778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:36.448394 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22730389-a335-499e-972c-aaa866418776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.449050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22730389-a335-499e-972c-aaa866418776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.449360 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22730389-a335-499e-972c-aaa866418776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.449632 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22730389-a335-499e-972c-aaa866418776 admin admin] PATH_INFO: `/v3/projects/cc100efb2e26474aa6590c71abddcc40` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.450839 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22730389-a335-499e-972c-aaa866418776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc100efb2e26474aa6590c71abddcc40)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.452779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22730389-a335-499e-972c-aaa866418776 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:36.461609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1357/2747] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:36.487730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:36.504160 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-22730389-a335-499e-972c-aaa866418776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc100efb2e26474aa6590c71abddcc40', 'request_id': 'req-22730389-a335-499e-972c-aaa866418776'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:36.510933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-22730389-a335-499e-972c-aaa866418776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc100efb2e26474aa6590c71abddcc40', 'request_id': 'req-22730389-a335-499e-972c-aaa866418776'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:36.518924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-22730389-a335-499e-972c-aaa866418776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc100efb2e26474aa6590c71abddcc40', 'request_id': 'req-22730389-a335-499e-972c-aaa866418776'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:36.532883 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-22730389-a335-499e-972c-aaa866418776 admin admin] The token cache is being invalidate because project cc100efb2e26474aa6590c71abddcc40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:36.533172 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-22730389-a335-499e-972c-aaa866418776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:36.533885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-22730389-a335-499e-972c-aaa866418776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:36.535337 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1391/2748] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:36 2022] DELETE /identity/v3/projects/cc100efb2e26474aa6590c71abddcc40 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:36.540296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f987f-798d-4dc8-b33b-870e6ce21b4b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.540584 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f987f-798d-4dc8-b33b-870e6ce21b4b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.540890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f987f-798d-4dc8-b33b-870e6ce21b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.560913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-922f987f-798d-4dc8-b33b-870e6ce21b4b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:36.562151 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-922f987f-798d-4dc8-b33b-870e6ce21b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:36.624261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1358/2749] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:36.632216 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5126db7-d9a0-44f0-8724-fe58d99f60b0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.632501 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5126db7-d9a0-44f0-8724-fe58d99f60b0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.632777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5126db7-d9a0-44f0-8724-fe58d99f60b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.652144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a5126db7-d9a0-44f0-8724-fe58d99f60b0 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:36.653386 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a5126db7-d9a0-44f0-8724-fe58d99f60b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:36.662827 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1392/2750] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:36.667043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f987f-798d-4dc8-b33b-870e6ce21b4b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.677948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f987f-798d-4dc8-b33b-870e6ce21b4b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.682775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.683391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.683658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.683930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.684570 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:36.684867 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.686493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:36.698108 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1359/2751] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:36.702430 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5126db7-d9a0-44f0-8724-fe58d99f60b0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.712099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5126db7-d9a0-44f0-8724-fe58d99f60b0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.716777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.717396 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.717673 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.717941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.718567 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.721086 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:36.750391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1393/2752] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:36 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:36.754478 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.759667 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:36.767499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362696a9-19c2-4342-8ea1-54372719d26a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.826480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.827092 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.827368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.827638 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.828273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.830720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:36.853079 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1360/2753] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:36 2022] POST /identity/v3/users => generated 305 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:36.857455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.868356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36a04d-347e-41f2-a6f1-972f395088be admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.872938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.873558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.873834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.874100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.874725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.876335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:36.886270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1394/2754] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:36.890220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.901872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cd430-7220-484e-ab89-54e37e0cdf94 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.907588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.908013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.908265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.908527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.909159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.910811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:36.928942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1361/2755] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:36.929107 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.940125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3604a8-788b-4f39-9356-d7837b37eb69 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.950813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:36.950813 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:36.961881 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a0bf3-7fce-416c-8885-7cf55f310a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.962573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a0bf3-7fce-416c-8885-7cf55f310a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.962873 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a0bf3-7fce-416c-8885-7cf55f310a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.963145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a0bf3-7fce-416c-8885-7cf55f310a42 admin admin] PATH_INFO: `/v3/projects/892eee90b8df4f5dbb5f18e89e7b0605/users/8dfbb01b14f24914b78cbc87e2907c19/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.965282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3503d67-538b-4545-8dfb-87c85b84756a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:36.971777 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:36.972458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:36.972728 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:36.973011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] PATH_INFO: `/v3/users/ea20af879c5647208871f2114e1b77ee` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:36.974166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea20af879c5647208871f2114e1b77ee)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.976252 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:36.983594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a0bf3-7fce-416c-8885-7cf55f310a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892eee90b8df4f5dbb5f18e89e7b0605, user_id=8dfbb01b14f24914b78cbc87e2907c19, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:36.986594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a0bf3-7fce-416c-8885-7cf55f310a42 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.004077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1395/2756] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:36 2022] PUT /identity/v3/projects/892eee90b8df4f5dbb5f18e89e7b0605/users/8dfbb01b14f24914b78cbc87e2907c19/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:37.018372 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.019168 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.020008 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.039308 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea20af879c5647208871f2114e1b77ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.053532 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea20af879c5647208871f2114e1b77ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.060798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] MFA Rules not processed for user `8dfbb01b14f24914b78cbc87e2907c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:37.061394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea20af879c5647208871f2114e1b77ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.062334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.075457 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea20af879c5647208871f2114e1b77ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.078558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccf2b6ea-3fe9-4aab-8f7a-721cf4c69d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea20af879c5647208871f2114e1b77ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.092408 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1362/2757] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:36 2022] DELETE /identity/v3/users/ea20af879c5647208871f2114e1b77ee => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:37.095033 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.095325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.095599 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.117786 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:37.118946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.149607 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1396/2758] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:37.154069 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.155822 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.181000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:37.198393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1363/2759] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:37.208339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.210259 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.213510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:37.244378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34935cc-f2f9-44e7-ad1e-5578bbbfb35e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.251429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.252042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.252590 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.252861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.253516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.256548 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.280309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad79cf-b149-427d-ab73-2752a663cbeb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.284578 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.285297 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.285651 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.285968 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.288908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1397/2760] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/projects => generated 409 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:37.293461 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.300099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.301561 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.314206 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1364/2761] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:04:37.371256 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cac3f-7eaf-4ec0-b9f9-56359ab37cac admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.377226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.377849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.378130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.378407 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.379165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.381230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.409604 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1398/2762] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/users => generated 311 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:37.411679 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.425060 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5107ab-8781-4968-a350-d4dcf67446ed service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.430339 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.430902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.431255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.431607 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.432314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.434023 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.449830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1365/2763] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:37.451048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.465814 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539df5-4fdd-457a-b3cf-04986696c992 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.471643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.472209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.472519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.472759 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] PATH_INFO: `/v3/projects/44707723a6b84787a82f8fa8d06c6f6b/users/d650fbbe74c34a91a1906b3d7e8d76c4/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.492624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44707723a6b84787a82f8fa8d06c6f6b, user_id=d650fbbe74c34a91a1906b3d7e8d76c4, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.496105 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.518156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1399/2764] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:37 2022] PUT /identity/v3/projects/44707723a6b84787a82f8fa8d06c6f6b/users/d650fbbe74c34a91a1906b3d7e8d76c4/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:37.519344 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.535775 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4383a08-6d3e-4dce-9c7a-a72a00ed88ec admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.649998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.650818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.651142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.651490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.652144 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.653899 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.664479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.666450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.674398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1366/2765] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:37.680601 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.680794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ef7fc-0193-4684-a55a-8a3b814be6b0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.685769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.686682 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.687169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.688153 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] PATH_INFO: `/v3/projects/1d18f2167d98453e9d8d56619b7e53b7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.690284 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d18f2167d98453e9d8d56619b7e53b7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.692469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.709487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f98d-432c-4ebd-851d-1bbb82fb2135 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.715351 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace4f13-4d99-4c82-b4a8-9768719727de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.716071 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4f13-4d99-4c82-b4a8-9768719727de admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.716435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4f13-4d99-4c82-b4a8-9768719727de admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.716721 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4f13-4d99-4c82-b4a8-9768719727de admin admin] PATH_INFO: `/v3/projects/44707723a6b84787a82f8fa8d06c6f6b/users/d650fbbe74c34a91a1906b3d7e8d76c4/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.719322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace4f13-4d99-4c82-b4a8-9768719727de admin admin] RBAC: Authorizing `identity:create_grant(project_id=44707723a6b84787a82f8fa8d06c6f6b, user_id=d650fbbe74c34a91a1906b3d7e8d76c4, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.722018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace4f13-4d99-4c82-b4a8-9768719727de admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.736989 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1367/2766] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:37 2022] PUT /identity/v3/projects/44707723a6b84787a82f8fa8d06c6f6b/users/d650fbbe74c34a91a1906b3d7e8d76c4/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:37.748218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.748563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.748906 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.754329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d18f2167d98453e9d8d56619b7e53b7', 'request_id': 'req-7dcf15e7-5272-4568-a74e-7c8fea2d2577'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.762220 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d18f2167d98453e9d8d56619b7e53b7', 'request_id': 'req-7dcf15e7-5272-4568-a74e-7c8fea2d2577'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.770423 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d18f2167d98453e9d8d56619b7e53b7', 'request_id': 'req-7dcf15e7-5272-4568-a74e-7c8fea2d2577'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.779928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] MFA Rules not processed for user `d650fbbe74c34a91a1906b3d7e8d76c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:37.782132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.787537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] The token cache is being invalidate because project 1d18f2167d98453e9d8d56619b7e53b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:37.788003 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.789333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7dcf15e7-5272-4568-a74e-7c8fea2d2577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:37.791095 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1400/2767] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:37 2022] DELETE /identity/v3/projects/1d18f2167d98453e9d8d56619b7e53b7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:37.795918 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63341b85-5c10-4bdb-be63-0ee64b482540 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.796446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63341b85-5c10-4bdb-be63-0ee64b482540 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.796732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63341b85-5c10-4bdb-be63-0ee64b482540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.820681 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-63341b85-5c10-4bdb-be63-0ee64b482540 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:37.822084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-63341b85-5c10-4bdb-be63-0ee64b482540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.834593 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:37.857296 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:37.874808 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1368/2768] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:37.888531 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.889988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.899592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1401/2769] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:37.908084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97162-f105-46ba-b58e-5404c13e8f88 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.908380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97162-f105-46ba-b58e-5404c13e8f88 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.908649 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97162-f105-46ba-b58e-5404c13e8f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.930836 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-16b97162-f105-46ba-b58e-5404c13e8f88 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:37.932042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16b97162-f105-46ba-b58e-5404c13e8f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:37.943112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1402/2770] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:37.944187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97162-f105-46ba-b58e-5404c13e8f88 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.953128 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97162-f105-46ba-b58e-5404c13e8f88 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.957379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93eb76f-23cd-4f65-bec0-c7735bbde0df None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.957867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.958451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.958727 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.958998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.959630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:37.959930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.961309 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.961707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.961973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.962260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.962530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.973515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.976937 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1403/2771] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:37.977173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:37.977953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:37.986560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1369/2772] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:37 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:37.988194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f639d1d-aa0b-4c11-b995-bc90e7b9bc65 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:37.992742 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:37.993500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:37.993665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:37.993905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:37.994539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:37.996467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.026338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1404/2773] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:37 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:38.034362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.045471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfc539-8107-4bb4-8ebf-35e9d730463d service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.109462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.110700 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.111065 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.111359 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.111975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.113936 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.139554 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1370/2774] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:38 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:38.143942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.152438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.159109 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfbe03-f456-450f-86f5-df5ce2bc2fb0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.164867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.165717 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.166088 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.166800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.167495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.169820 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.181246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1405/2775] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:38.185597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.199512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77339c-cba1-424c-be48-67377bb1034e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.206257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.207356 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.207887 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.208367 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.209281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.211275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.223333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1406/2776] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:38.227298 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.231527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a364a-ca05-4014-bdd9-d542cc3337bd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.236830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.237571 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.237857 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.238135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.239902 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b517be-432c-464d-ab8a-22900be80b87 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.240212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:38.245630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba50e97-be63-4fcf-8941-c11017610c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.246363 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba50e97-be63-4fcf-8941-c11017610c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.246751 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba50e97-be63-4fcf-8941-c11017610c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.247042 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba50e97-be63-4fcf-8941-c11017610c94 admin admin] PATH_INFO: `/v3/projects/0763b53982f0438c8551cf03da471f32/users/0ee7e64b51704a06823b20fcb05521b2/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.278473 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba50e97-be63-4fcf-8941-c11017610c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0763b53982f0438c8551cf03da471f32, user_id=0ee7e64b51704a06823b20fcb05521b2, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.281100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba50e97-be63-4fcf-8941-c11017610c94 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.301805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:38.305720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1407/2777] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:38 2022] PUT /identity/v3/projects/0763b53982f0438c8551cf03da471f32/users/0ee7e64b51704a06823b20fcb05521b2/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:38.311542 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a505f-254b-46a9-80c9-2ac0cb39032a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.312442 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a505f-254b-46a9-80c9-2ac0cb39032a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.312705 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a505f-254b-46a9-80c9-2ac0cb39032a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.326380 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.327974 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.338474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1371/2778] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:38.361385 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ec1a505f-254b-46a9-80c9-2ac0cb39032a None None] MFA Rules not processed for user `0ee7e64b51704a06823b20fcb05521b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:38.363750 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ec1a505f-254b-46a9-80c9-2ac0cb39032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:38.462783 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1408/2779] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:38.464149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.481907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71340-3641-40ce-bb37-c11c571aaa22 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.554237 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.554843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.555128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.555395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.556035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.558034 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.593517 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1372/2780] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:38 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:38.597285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a505f-254b-46a9-80c9-2ac0cb39032a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.611071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a505f-254b-46a9-80c9-2ac0cb39032a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.625505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:38.679497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.680254 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.680611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.681048 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.681852 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.684138 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.721663 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1409/2781] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:38 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:38.723527 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.736752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a909ee-3315-4c9c-86cb-4a1c507102b9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.742588 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.743204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.743486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.743780 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.744521 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.747075 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.760117 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1373/2782] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:38.764399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.776276 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4dbe0-28a7-4ce0-9109-e25f9c9bbc86 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.781103 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.782049 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.782338 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.782608 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] PATH_INFO: `/v3/projects/d1234c71065a4cb29f9038da2b5b0304/users/2a62cd25ea204f288d4f850c6a9359e0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.798671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1234c71065a4cb29f9038da2b5b0304, user_id=2a62cd25ea204f288d4f850c6a9359e0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.801453 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.815915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1410/2783] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:38 2022] PUT /identity/v3/projects/d1234c71065a4cb29f9038da2b5b0304/users/2a62cd25ea204f288d4f850c6a9359e0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:38.820025 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.831224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9e11c-d7cf-4da4-878b-a4614b5e7305 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.887937 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114f934-ea33-4794-aa23-ffb811959493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.888548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114f934-ea33-4794-aa23-ffb811959493 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.888825 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114f934-ea33-4794-aa23-ffb811959493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.889097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114f934-ea33-4794-aa23-ffb811959493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.890642 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114f934-ea33-4794-aa23-ffb811959493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.892712 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114f934-ea33-4794-aa23-ffb811959493 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.905627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1374/2784] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:38.907332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:38.917621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9110a-476c-4dd6-8faf-97d03871edc4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:38.923361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:38.924015 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.924300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.924690 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] PATH_INFO: `/v3/projects/d1234c71065a4cb29f9038da2b5b0304/users/2a62cd25ea204f288d4f850c6a9359e0/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.926767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1234c71065a4cb29f9038da2b5b0304, user_id=2a62cd25ea204f288d4f850c6a9359e0, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:38.929859 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:38.942005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1411/2785] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:38 2022] PUT /identity/v3/projects/d1234c71065a4cb29f9038da2b5b0304/users/2a62cd25ea204f288d4f850c6a9359e0/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:38.952897 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096327-2c25-4d26-9fa8-a4da3fbd805d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:38.953331 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096327-2c25-4d26-9fa8-a4da3fbd805d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:38.953609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096327-2c25-4d26-9fa8-a4da3fbd805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:38.984751 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-30096327-2c25-4d26-9fa8-a4da3fbd805d None None] MFA Rules not processed for user `2a62cd25ea204f288d4f850c6a9359e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:38.986015 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-30096327-2c25-4d26-9fa8-a4da3fbd805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:39.030055 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:39.067575 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1375/2786] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:39.079011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:39.136047 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea25d0b-3e1c-4329-b1f9-60d68e03e3ca admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:39.139919 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67cf4d-9a69-479d-b3bc-cfc6bcffbe2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:39.140577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67cf4d-9a69-479d-b3bc-cfc6bcffbe2e service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:39.140863 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67cf4d-9a69-479d-b3bc-cfc6bcffbe2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:39.141156 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67cf4d-9a69-479d-b3bc-cfc6bcffbe2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:39.150505 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67cf4d-9a69-479d-b3bc-cfc6bcffbe2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:39.151953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67cf4d-9a69-479d-b3bc-cfc6bcffbe2e service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:39.162617 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1412/2787] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:40.628460 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30096327-2c25-4d26-9fa8-a4da3fbd805d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:40.693797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30096327-2c25-4d26-9fa8-a4da3fbd805d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:40.698577 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:40.699208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:40.699648 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:40.699926 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] PATH_INFO: `/v3/users/0ee7e64b51704a06823b20fcb05521b2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:40.701070 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee7e64b51704a06823b20fcb05521b2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:40.703019 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:40.751523 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee7e64b51704a06823b20fcb05521b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:40.763142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee7e64b51704a06823b20fcb05521b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:40.770362 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee7e64b51704a06823b20fcb05521b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:40.777459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee7e64b51704a06823b20fcb05521b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:40.778069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee7e64b51704a06823b20fcb05521b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:40.784901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1376/2788] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:40 2022] DELETE /identity/v3/users/0ee7e64b51704a06823b20fcb05521b2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:40.789446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348241c-e03a-4c48-901b-a9990610808e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:40.789907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348241c-e03a-4c48-901b-a9990610808e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:40.790032 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348241c-e03a-4c48-901b-a9990610808e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:40.808509 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1348241c-e03a-4c48-901b-a9990610808e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:40.809743 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1348241c-e03a-4c48-901b-a9990610808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:40.866677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1413/2789] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:40.876174 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:40.877735 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:40.935487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2402c-b499-4ed8-8224-fb8afbbe8d62 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:40.939672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:40.940274 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:40.940555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:40.940827 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:40.949761 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:40.951182 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:40.955652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:40.962878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1377/2790] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:41.232016 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348241c-e03a-4c48-901b-a9990610808e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:41.233592 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1348241c-e03a-4c48-901b-a9990610808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:41.245895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348241c-e03a-4c48-901b-a9990610808e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:41.251084 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:41.251713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:41.251985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:41.252327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] PATH_INFO: `/v3/projects/0763b53982f0438c8551cf03da471f32` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:41.253886 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0763b53982f0438c8551cf03da471f32)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:41.255795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:41.264433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:41.304433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0763b53982f0438c8551cf03da471f32', 'request_id': 'req-0f0d3657-3d64-4879-a8a1-518e198c96f1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.310638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0763b53982f0438c8551cf03da471f32', 'request_id': 'req-0f0d3657-3d64-4879-a8a1-518e198c96f1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.317546 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0763b53982f0438c8551cf03da471f32', 'request_id': 'req-0f0d3657-3d64-4879-a8a1-518e198c96f1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.329679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] The token cache is being invalidate because project 0763b53982f0438c8551cf03da471f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:41.329994 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.330611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.332116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1414/2791] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:41 2022] DELETE /identity/v3/projects/0763b53982f0438c8551cf03da471f32 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:41.336067 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:41.345548 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c337d0-d16e-41b5-9b04-6d7fe5f60ee4 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:41.400994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:41.401581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:41.401859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:41.402127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] PATH_INFO: `/v3/users/2a62cd25ea204f288d4f850c6a9359e0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:41.403227 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a62cd25ea204f288d4f850c6a9359e0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:41.405120 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:41.451238 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a62cd25ea204f288d4f850c6a9359e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.462833 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a62cd25ea204f288d4f850c6a9359e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.469414 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a62cd25ea204f288d4f850c6a9359e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.476386 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a62cd25ea204f288d4f850c6a9359e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.477005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-32cb78b6-ab89-4e45-9f01-7592818fa87b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a62cd25ea204f288d4f850c6a9359e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.482693 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1378/2792] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:41 2022] DELETE /identity/v3/users/2a62cd25ea204f288d4f850c6a9359e0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:41.753613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:41.755271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:41.770956 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d3657-3d64-4879-a8a1-518e198c96f1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:41.831848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:41.832712 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:41.833089 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:41.833482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] PATH_INFO: `/v3/projects/d1234c71065a4cb29f9038da2b5b0304` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:41.834761 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1234c71065a4cb29f9038da2b5b0304)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:41.836877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:41.888844 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1234c71065a4cb29f9038da2b5b0304', 'request_id': 'req-b174f13b-8664-4ae5-baba-cdefbb90655f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.895271 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1234c71065a4cb29f9038da2b5b0304', 'request_id': 'req-b174f13b-8664-4ae5-baba-cdefbb90655f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.900856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1234c71065a4cb29f9038da2b5b0304', 'request_id': 'req-b174f13b-8664-4ae5-baba-cdefbb90655f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.914474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] The token cache is being invalidate because project d1234c71065a4cb29f9038da2b5b0304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:41.914780 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.915380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b174f13b-8664-4ae5-baba-cdefbb90655f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:41.916849 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1415/2793] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:41 2022] DELETE /identity/v3/projects/d1234c71065a4cb29f9038da2b5b0304 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:41.921950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80e6f-90a8-4d6f-826d-50029e94af7f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:41.922984 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80e6f-90a8-4d6f-826d-50029e94af7f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:41.923397 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80e6f-90a8-4d6f-826d-50029e94af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:41.942860 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-20f80e6f-90a8-4d6f-826d-50029e94af7f None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:41.944035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-20f80e6f-90a8-4d6f-826d-50029e94af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:42.003121 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1379/2794] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:42.010942 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd0bef-69b1-4438-92f6-00528eb006a3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.011229 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd0bef-69b1-4438-92f6-00528eb006a3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.011502 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd0bef-69b1-4438-92f6-00528eb006a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.030925 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e8bd0bef-69b1-4438-92f6-00528eb006a3 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:42.032921 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e8bd0bef-69b1-4438-92f6-00528eb006a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:42.041174 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1416/2795] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:42.045468 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f80e6f-90a8-4d6f-826d-50029e94af7f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.056197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f80e6f-90a8-4d6f-826d-50029e94af7f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.061042 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.061676 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.061958 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.062230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.062902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:42.063202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.064774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.079172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1380/2796] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:04:42.081455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd0bef-69b1-4438-92f6-00528eb006a3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.090736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd0bef-69b1-4438-92f6-00528eb006a3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.096621 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.097188 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.097517 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.097773 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.098391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.100611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.120966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:42.130355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1417/2797] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:42 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:42.134862 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.141787 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:42.145538 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fad2c-714e-497b-a1bb-231fb775fb81 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.202343 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.202890 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.203168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.203438 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.204583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.206516 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.227263 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1381/2798] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:42 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:42.231368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.241594 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d2675-93c1-4ce8-9eca-788e76c810c1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.246245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.246830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.247110 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.247383 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.248021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.249715 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.259656 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1418/2799] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:42.263550 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.273709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53978637-4640-4d26-a02b-d4fd04c9ab96 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.278246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714b118-a4be-4394-b95f-37c96dacdfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.278826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714b118-a4be-4394-b95f-37c96dacdfa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.279101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714b118-a4be-4394-b95f-37c96dacdfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.279369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714b118-a4be-4394-b95f-37c96dacdfa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.279991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714b118-a4be-4394-b95f-37c96dacdfa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.282391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714b118-a4be-4394-b95f-37c96dacdfa8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.292534 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1382/2800] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:42.296565 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.306333 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400b30e-cef7-4a56-93db-8a5d7a93f045 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.310936 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.311569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.311847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.312144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] PATH_INFO: `/v3/projects/f2b0c7b3ec054715a231493c2494880a/users/1b3edef5104a4786af16ad312d35e607/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.326236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b0c7b3ec054715a231493c2494880a, user_id=1b3edef5104a4786af16ad312d35e607, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.329720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.343566 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1419/2801] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:42 2022] PUT /identity/v3/projects/f2b0c7b3ec054715a231493c2494880a/users/1b3edef5104a4786af16ad312d35e607/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:42.354273 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc888f4-a688-4ac7-b030-ead00025a02f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.354580 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc888f4-a688-4ac7-b030-ead00025a02f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.354849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc888f4-a688-4ac7-b030-ead00025a02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.381665 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1fc888f4-a688-4ac7-b030-ead00025a02f None None] MFA Rules not processed for user `1b3edef5104a4786af16ad312d35e607`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:42.382846 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1fc888f4-a688-4ac7-b030-ead00025a02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:42.445370 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1383/2802] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:42.449947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.509369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b661bd-3fff-464d-becd-d255b10595eb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.513875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.514456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.514731 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.514996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.515613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.518126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.544467 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1420/2803] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:42 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:42.548598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc888f4-a688-4ac7-b030-ead00025a02f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.559568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc888f4-a688-4ac7-b030-ead00025a02f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.619319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.619925 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.620198 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.620465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.621081 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.623001 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.645288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1384/2804] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:42 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:42.649336 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.660379 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2836f2-c2cc-49c2-ace3-e34cda619327 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.665100 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.665756 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.666034 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.666304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.666933 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.668545 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.679071 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1421/2805] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:42.683224 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.688738 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:42.694988 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19cdd7-995c-41f3-94c6-32f3fc4c17a2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.699894 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.700563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.700861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.701126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] PATH_INFO: `/v3/projects/cb32fb09145b4cf7825a7a5f5b79b220/users/704512e8365f471694f166a98803bec1/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.716363 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb32fb09145b4cf7825a7a5f5b79b220, user_id=704512e8365f471694f166a98803bec1, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.719018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.732243 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1385/2806] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:42 2022] PUT /identity/v3/projects/cb32fb09145b4cf7825a7a5f5b79b220/users/704512e8365f471694f166a98803bec1/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:42.736741 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.746544 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc01a8-b50e-49a4-8ae4-c552d8fd52b7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.801462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c7ee4-34d7-445c-8165-ace02afc9095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.802010 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c7ee4-34d7-445c-8165-ace02afc9095 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.802285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c7ee4-34d7-445c-8165-ace02afc9095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.802551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c7ee4-34d7-445c-8165-ace02afc9095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.803175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c7ee4-34d7-445c-8165-ace02afc9095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.804763 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c7ee4-34d7-445c-8165-ace02afc9095 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.814266 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1422/2807] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:42.818467 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.828479 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d081c73-e9df-4334-966a-de24b42832bc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:42.833639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:42.834287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.834597 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.834872 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] PATH_INFO: `/v3/projects/cb32fb09145b4cf7825a7a5f5b79b220/users/704512e8365f471694f166a98803bec1/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.836796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb32fb09145b4cf7825a7a5f5b79b220, user_id=704512e8365f471694f166a98803bec1, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:42.839462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:42.850163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1386/2808] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:42 2022] PUT /identity/v3/projects/cb32fb09145b4cf7825a7a5f5b79b220/users/704512e8365f471694f166a98803bec1/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:42.860314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:42.860831 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:42.860941 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:42.888428 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] MFA Rules not processed for user `704512e8365f471694f166a98803bec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:42.889732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:42.911938 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:42.963021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1423/2809] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:42.974613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:42.976463 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:43.037595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717cfd1-9945-422d-950a-b512ace981e5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:43.041275 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb4520-7d69-4164-879d-530940abdae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:43.041889 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb4520-7d69-4164-879d-530940abdae5 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:43.042172 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb4520-7d69-4164-879d-530940abdae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:43.042446 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb4520-7d69-4164-879d-530940abdae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:43.111322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb4520-7d69-4164-879d-530940abdae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:43.120462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb4520-7d69-4164-879d-530940abdae5 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:43.131225 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1387/2810] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:45.882055 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:45.883477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:45.945888 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832b431-c7d1-4ae7-98f3-0cd6f3fc6477 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:45.950329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:45.950943 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:45.951218 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:45.951484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] PATH_INFO: `/v3/users/8dfbb01b14f24914b78cbc87e2907c19` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:45.952581 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfbb01b14f24914b78cbc87e2907c19)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:45.954551 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:46.000142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfbb01b14f24914b78cbc87e2907c19'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.007533 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfbb01b14f24914b78cbc87e2907c19'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.019777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfbb01b14f24914b78cbc87e2907c19'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.020477 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfbb01b14f24914b78cbc87e2907c19'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.027891 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfbb01b14f24914b78cbc87e2907c19'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.034155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1424/2811] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:45 2022] DELETE /identity/v3/users/8dfbb01b14f24914b78cbc87e2907c19 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:46.038660 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:46.039111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:46.039250 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:46.059756 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:46.060985 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:46.118267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1388/2812] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:46.128639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:46.130056 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:46.174520 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:46.191720 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca331f0-44fa-4a95-8ee0-2804189cfb95 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:46.195299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:46.195892 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:46.196166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:46.196433 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:46.205108 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:46.206538 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:46.215726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1425/2813] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:46.486229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:46.487725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:46.500413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1659e-ec9f-4870-809b-8d93b3741e2d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:46.505388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:46.506242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:46.506582 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:46.506962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] PATH_INFO: `/v3/projects/892eee90b8df4f5dbb5f18e89e7b0605` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:46.508378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=892eee90b8df4f5dbb5f18e89e7b0605)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:46.510377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:46.531219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:46.565987 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892eee90b8df4f5dbb5f18e89e7b0605', 'request_id': 'req-ca68a980-4c74-4b08-8d41-dc25b504f468'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.572288 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892eee90b8df4f5dbb5f18e89e7b0605', 'request_id': 'req-ca68a980-4c74-4b08-8d41-dc25b504f468'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.578184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892eee90b8df4f5dbb5f18e89e7b0605', 'request_id': 'req-ca68a980-4c74-4b08-8d41-dc25b504f468'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.591995 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] The token cache is being invalidate because project 892eee90b8df4f5dbb5f18e89e7b0605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:46.592283 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.592924 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.595163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1389/2814] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:46 2022] DELETE /identity/v3/projects/892eee90b8df4f5dbb5f18e89e7b0605 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:46.599265 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:46.609905 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6288a5e-0f76-4afc-a8ef-c0cf1acdc5be service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:46.670800 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:46.671568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:46.671882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:46.672099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] PATH_INFO: `/v3/users/d650fbbe74c34a91a1906b3d7e8d76c4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:46.673303 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d650fbbe74c34a91a1906b3d7e8d76c4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:46.675238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:46.721875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd650fbbe74c34a91a1906b3d7e8d76c4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.728537 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd650fbbe74c34a91a1906b3d7e8d76c4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.740132 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd650fbbe74c34a91a1906b3d7e8d76c4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.740866 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd650fbbe74c34a91a1906b3d7e8d76c4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.749082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-adaedb9c-53d9-45b2-ac19-4748abe62052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd650fbbe74c34a91a1906b3d7e8d76c4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:46.755843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1426/2815] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:46 2022] DELETE /identity/v3/users/d650fbbe74c34a91a1906b3d7e8d76c4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:47.026481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.027930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:47.043329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68a980-4c74-4b08-8d41-dc25b504f468 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.101426 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.101980 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.102255 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.102525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] PATH_INFO: `/v3/projects/44707723a6b84787a82f8fa8d06c6f6b` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.103622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44707723a6b84787a82f8fa8d06c6f6b)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.105569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.153188 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44707723a6b84787a82f8fa8d06c6f6b', 'request_id': 'req-748554de-17de-4f41-9fb0-a53e5f3c6ed1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:47.159944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44707723a6b84787a82f8fa8d06c6f6b', 'request_id': 'req-748554de-17de-4f41-9fb0-a53e5f3c6ed1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:47.165441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44707723a6b84787a82f8fa8d06c6f6b', 'request_id': 'req-748554de-17de-4f41-9fb0-a53e5f3c6ed1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:47.177913 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] The token cache is being invalidate because project 44707723a6b84787a82f8fa8d06c6f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:47.178229 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:47.178859 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-748554de-17de-4f41-9fb0-a53e5f3c6ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:47.180322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1390/2816] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:47 2022] DELETE /identity/v3/projects/44707723a6b84787a82f8fa8d06c6f6b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:47.185114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5c0e7-6f66-46cb-ae0f-a65b8db75ff8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.185624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5c0e7-6f66-46cb-ae0f-a65b8db75ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.185736 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5c0e7-6f66-46cb-ae0f-a65b8db75ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.206597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-feb5c0e7-6f66-46cb-ae0f-a65b8db75ff8 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:47.207788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-feb5c0e7-6f66-46cb-ae0f-a65b8db75ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:47.262489 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:47.272615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1427/2817] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:04:47.280506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3cf07-912e-403c-a8cc-74528702229e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.280774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3cf07-912e-403c-a8cc-74528702229e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.281044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3cf07-912e-403c-a8cc-74528702229e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.301099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f7b3cf07-912e-403c-a8cc-74528702229e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:47.302325 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f7b3cf07-912e-403c-a8cc-74528702229e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:47.310450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1391/2818] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:47.315068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5c0e7-6f66-46cb-ae0f-a65b8db75ff8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.325483 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5c0e7-6f66-46cb-ae0f-a65b8db75ff8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.330114 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.330702 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.330978 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.331247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.331877 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:04:47.332175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.333804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.344980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1428/2819] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 15:04:47.349168 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3cf07-912e-403c-a8cc-74528702229e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.358496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3cf07-912e-403c-a8cc-74528702229e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.363543 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.364132 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.364413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.364691 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.365364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.367246 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.393797 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1392/2820] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:47 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:47.397858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.407874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e5f1-e82b-43f2-95f1-e9fd4ed4753e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.462329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.462924 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.463203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.463474 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.464152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.466197 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.490345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1429/2821] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:47 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:47.491563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.498244 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:47.503536 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8198ef-9ea7-4b49-8a7a-ae7220a66ce6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.510135 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.510884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.511186 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.511673 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.512356 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.514569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.525169 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1393/2822] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:47.529129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.539371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26ac22-d0b7-466f-a393-af636946bd8f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.543976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac4867-d5cf-4a2a-85e5-770bafb59167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.544591 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4867-d5cf-4a2a-85e5-770bafb59167 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.544870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4867-d5cf-4a2a-85e5-770bafb59167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.545145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4867-d5cf-4a2a-85e5-770bafb59167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.545811 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac4867-d5cf-4a2a-85e5-770bafb59167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.547392 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac4867-d5cf-4a2a-85e5-770bafb59167 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.559577 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1430/2823] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:04:47.562853 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.575871 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d20d9-ed79-4572-96ae-d6b24db69da1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.581193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.581849 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.582123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.582387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] PATH_INFO: `/v3/projects/00b4539bcea2422c87cdab8f0f570f82/users/9431d468122d43bd985ac4a52b7f5e8a/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.596377 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b4539bcea2422c87cdab8f0f570f82, user_id=9431d468122d43bd985ac4a52b7f5e8a, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.599752 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.614106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1394/2824] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:47 2022] PUT /identity/v3/projects/00b4539bcea2422c87cdab8f0f570f82/users/9431d468122d43bd985ac4a52b7f5e8a/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:47.684818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.685081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.685389 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.714378 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] MFA Rules not processed for user `9431d468122d43bd985ac4a52b7f5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:47.715633 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:47.802878 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1431/2825] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:47.805680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.822867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.824420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:47.886409 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedc476-dd00-499c-a9a7-242bbf708e2e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.888184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0dc24-46d5-4b24-b6e1-db8cd930d6c6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:47.892600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.893228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.893518 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.893788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.893893 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:47.895628 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:47.896057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:47.896190 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:47.896277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:47.897076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.899208 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:47.907357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:47.937355 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1395/2826] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:04:47 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:04:47.937724 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:47.988451 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:47.994362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:48.007335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1432/2827] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:48.026620 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d558f8-b4ab-4d3b-9bdc-4d6e6fb46832 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:48.033483 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:48.034106 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:48.034401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:48.034653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:48.035529 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:48.037563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:48.065481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1396/2828] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:04:47 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:04:48.068063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:48.079510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fb148-539b-4516-8a1a-24cedb3d8355 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:48.085299 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:48.085974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:48.086311 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:48.086646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:48.087355 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:48.089063 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:48.104725 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1433/2829] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:48.108928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:48.119219 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f95fb-4b01-4dcb-9d3c-a0b614d61bab admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:48.128000 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:48.128729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:48.128991 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:48.129392 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] PATH_INFO: `/v3/projects/a220decf98424f05a7c4cbd5c9700250/users/932694a6bda64ef39e2f74ccfdf45534/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:48.148400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a220decf98424f05a7c4cbd5c9700250, user_id=932694a6bda64ef39e2f74ccfdf45534, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:48.151449 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:48.167938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1397/2830] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:48 2022] PUT /identity/v3/projects/a220decf98424f05a7c4cbd5c9700250/users/932694a6bda64ef39e2f74ccfdf45534/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:48.172081 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:48.182369 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab146c9-279d-4946-8a46-ad244b6cf978 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:48.251804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afad43a-ab41-4ef8-bbdc-b49c45400f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:48.252434 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afad43a-ab41-4ef8-bbdc-b49c45400f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:48.252724 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afad43a-ab41-4ef8-bbdc-b49c45400f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:48.252995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afad43a-ab41-4ef8-bbdc-b49c45400f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:48.253677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afad43a-ab41-4ef8-bbdc-b49c45400f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:48.255310 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afad43a-ab41-4ef8-bbdc-b49c45400f28 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:48.265948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1434/2831] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:04:48.270698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:48.281595 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc44765-c0e9-412a-9983-9aac60610712 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:48.287422 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:48.288267 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:48.288658 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:48.288788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] PATH_INFO: `/v3/projects/a220decf98424f05a7c4cbd5c9700250/users/932694a6bda64ef39e2f74ccfdf45534/roles/82245735988747a4bc5704727a1cf374` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:48.290901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a220decf98424f05a7c4cbd5c9700250, user_id=932694a6bda64ef39e2f74ccfdf45534, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:48.293649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:48.304680 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1398/2832] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:04:48 2022] PUT /identity/v3/projects/a220decf98424f05a7c4cbd5c9700250/users/932694a6bda64ef39e2f74ccfdf45534/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:48.315319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f7565-2ce8-4c24-bae9-6eaa0cff9246 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:48.315609 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f7565-2ce8-4c24-bae9-6eaa0cff9246 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:48.315885 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f7565-2ce8-4c24-bae9-6eaa0cff9246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:48.345541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f99f7565-2ce8-4c24-bae9-6eaa0cff9246 None None] MFA Rules not processed for user `932694a6bda64ef39e2f74ccfdf45534`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:48.346802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f99f7565-2ce8-4c24-bae9-6eaa0cff9246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:48.424039 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1435/2833] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:48.435083 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:48.446037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:48.495391 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c901-7fdf-4169-8c95-b8fcceb668c3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:48.499047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97982f7-8f0a-46c9-84a2-0b1a4c005a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:48.499643 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97982f7-8f0a-46c9-84a2-0b1a4c005a00 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:48.499923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97982f7-8f0a-46c9-84a2-0b1a4c005a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:48.500191 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97982f7-8f0a-46c9-84a2-0b1a4c005a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:48.509649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97982f7-8f0a-46c9-84a2-0b1a4c005a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:48.511062 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97982f7-8f0a-46c9-84a2-0b1a4c005a00 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:48.520831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1399/2834] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:49.058300 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f7565-2ce8-4c24-bae9-6eaa0cff9246 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:49.077969 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:49.123408 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f7565-2ce8-4c24-bae9-6eaa0cff9246 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:49.127930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:49.128552 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:49.128838 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:49.129112 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] PATH_INFO: `/v3/users/9431d468122d43bd985ac4a52b7f5e8a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:49.130255 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9431d468122d43bd985ac4a52b7f5e8a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:49.132164 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:49.176065 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9431d468122d43bd985ac4a52b7f5e8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.182754 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9431d468122d43bd985ac4a52b7f5e8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.194818 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9431d468122d43bd985ac4a52b7f5e8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.195478 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9431d468122d43bd985ac4a52b7f5e8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.202770 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9431d468122d43bd985ac4a52b7f5e8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.209805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1436/2835] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:49 2022] DELETE /identity/v3/users/9431d468122d43bd985ac4a52b7f5e8a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:49.214389 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:49.214895 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:49.215011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:49.233374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:04:49.234583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:49.291999 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1400/2836] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:04:49.301880 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:49.303375 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:49.359165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e28bda-6fd7-4615-a1b6-de004c054117 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:49.362846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:49.363510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:49.363789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:49.364064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:49.373113 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:49.374582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:49.384332 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1437/2837] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:04:49.648789 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:49.650329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:49.663458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a09b40-188a-4624-8063-18f956b71c47 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:49.668878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:49.669800 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:49.670148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:49.670480 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] PATH_INFO: `/v3/projects/00b4539bcea2422c87cdab8f0f570f82` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:49.671917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b4539bcea2422c87cdab8f0f570f82)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:49.674767 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:49.722261 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b4539bcea2422c87cdab8f0f570f82', 'request_id': 'req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.728938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b4539bcea2422c87cdab8f0f570f82', 'request_id': 'req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.734605 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b4539bcea2422c87cdab8f0f570f82', 'request_id': 'req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.747504 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] The token cache is being invalidate because project 00b4539bcea2422c87cdab8f0f570f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:49.747796 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.748420 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.749961 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1401/2838] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:49 2022] DELETE /identity/v3/projects/00b4539bcea2422c87cdab8f0f570f82 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:04:49.754569 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:49.764258 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a407-9ea0-4a14-967d-9aeeec827bc4 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:49.796406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:49.824203 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:49.824830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:49.825144 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:49.825455 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] PATH_INFO: `/v3/users/932694a6bda64ef39e2f74ccfdf45534` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:49.826571 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=932694a6bda64ef39e2f74ccfdf45534)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:49.828482 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:49.872420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932694a6bda64ef39e2f74ccfdf45534'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.880200 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932694a6bda64ef39e2f74ccfdf45534'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.891787 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932694a6bda64ef39e2f74ccfdf45534'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.892443 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932694a6bda64ef39e2f74ccfdf45534'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.899438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932694a6bda64ef39e2f74ccfdf45534'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:49.905179 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1438/2839] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:04:49 2022] DELETE /identity/v3/users/932694a6bda64ef39e2f74ccfdf45534 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:50.184656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:04:50.186137 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:04:50.201768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb308a4-f77a-4d27-ba83-8e97d5ed0453 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:04:50.233286 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:04:50.261009 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:04:50.261604 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:04:50.261887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:04:50.262163 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] PATH_INFO: `/v3/projects/a220decf98424f05a7c4cbd5c9700250` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:04:50.263405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a220decf98424f05a7c4cbd5c9700250)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:04:50.265361 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:04:50.310514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a220decf98424f05a7c4cbd5c9700250', 'request_id': 'req-8c97f871-caad-4722-add5-eb16fabb3acd'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:50.316473 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a220decf98424f05a7c4cbd5c9700250', 'request_id': 'req-8c97f871-caad-4722-add5-eb16fabb3acd'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:50.322018 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a220decf98424f05a7c4cbd5c9700250', 'request_id': 'req-8c97f871-caad-4722-add5-eb16fabb3acd'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:50.334822 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] The token cache is being invalidate because project a220decf98424f05a7c4cbd5c9700250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:04:50.335111 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:50.335725 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8c97f871-caad-4722-add5-eb16fabb3acd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:04:50.337218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1402/2840] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:04:50 2022] DELETE /identity/v3/projects/a220decf98424f05a7c4cbd5c9700250 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:04:59.901805 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:00.331834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:12.902183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:12.903585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:12.978292 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f43a2-846f-4b02-b6d6-75253b76c9b4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:13.034028 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:13.034739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:13.035573 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:13.035846 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] PATH_INFO: `/v3/users/1b3edef5104a4786af16ad312d35e607` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:13.036975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3edef5104a4786af16ad312d35e607)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:13.038930 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:13.091598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3edef5104a4786af16ad312d35e607'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.098840 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3edef5104a4786af16ad312d35e607'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.112231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3edef5104a4786af16ad312d35e607'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.112951 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3edef5104a4786af16ad312d35e607'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.119884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3edef5104a4786af16ad312d35e607'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.125860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1439/2841] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:05:12 2022] DELETE /identity/v3/users/1b3edef5104a4786af16ad312d35e607 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:05:13.130823 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:13.131321 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:13.131453 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:13.159317 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:13.160541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:13.219474 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1403/2842] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:13.229175 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:13.230627 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:13.251000 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:13.309721 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35f8d2-92d0-4353-9c6b-b27c48cde20b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:13.313361 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:13.313961 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:13.314238 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:13.314510 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:13.323964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:13.325444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:13.335328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1440/2843] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:05:13.596557 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:13.598027 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:13.611407 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb499d2-e734-4bcd-91ff-584ef925c8ee None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:13.616874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:13.617644 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:13.618016 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:13.618486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] PATH_INFO: `/v3/projects/f2b0c7b3ec054715a231493c2494880a` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:13.620139 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b0c7b3ec054715a231493c2494880a)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:13.622369 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:13.668831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b0c7b3ec054715a231493c2494880a', 'request_id': 'req-ae597729-4cca-4a16-8a20-d1094d2da6f6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.674896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b0c7b3ec054715a231493c2494880a', 'request_id': 'req-ae597729-4cca-4a16-8a20-d1094d2da6f6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.680783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b0c7b3ec054715a231493c2494880a', 'request_id': 'req-ae597729-4cca-4a16-8a20-d1094d2da6f6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.693482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] The token cache is being invalidate because project f2b0c7b3ec054715a231493c2494880a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:05:13.693768 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.694374 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.695842 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1404/2844] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:05:13 2022] DELETE /identity/v3/projects/f2b0c7b3ec054715a231493c2494880a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:05:13.699804 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:13.709446 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ead6-eedb-410c-99c2-d3f441cfb74d service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:13.766290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:13.766915 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:13.767194 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:13.767497 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] PATH_INFO: `/v3/users/704512e8365f471694f166a98803bec1` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:13.768640 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=704512e8365f471694f166a98803bec1)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:13.770672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:13.816713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '704512e8365f471694f166a98803bec1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.823513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '704512e8365f471694f166a98803bec1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.835129 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '704512e8365f471694f166a98803bec1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.835778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '704512e8365f471694f166a98803bec1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.842789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1a286532-f5fe-411e-bd92-2159e7504b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '704512e8365f471694f166a98803bec1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:13.849215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1441/2845] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:05:13 2022] DELETE /identity/v3/users/704512e8365f471694f166a98803bec1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:05:14.113698 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.115211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:14.129950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597729-4cca-4a16-8a20-d1094d2da6f6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.185312 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.185997 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.186204 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.186807 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] PATH_INFO: `/v3/projects/cb32fb09145b4cf7825a7a5f5b79b220` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.187618 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb32fb09145b4cf7825a7a5f5b79b220)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.189535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.210512 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:14.236485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb32fb09145b4cf7825a7a5f5b79b220', 'request_id': 'req-7504e2da-3f03-4035-8e7b-90c3116b2fbe'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:14.242525 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb32fb09145b4cf7825a7a5f5b79b220', 'request_id': 'req-7504e2da-3f03-4035-8e7b-90c3116b2fbe'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:14.247942 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb32fb09145b4cf7825a7a5f5b79b220', 'request_id': 'req-7504e2da-3f03-4035-8e7b-90c3116b2fbe'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:14.259975 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] The token cache is being invalidate because project cb32fb09145b4cf7825a7a5f5b79b220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:05:14.260260 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:14.260884 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7504e2da-3f03-4035-8e7b-90c3116b2fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:14.262563 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1405/2846] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:05:14 2022] DELETE /identity/v3/projects/cb32fb09145b4cf7825a7a5f5b79b220 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:05:14.268314 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1ae2c-8335-4c6c-b065-a8ce0c919f3b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.268795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1ae2c-8335-4c6c-b065-a8ce0c919f3b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.268895 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1ae2c-8335-4c6c-b065-a8ce0c919f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.303984 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:14.321890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-24e1ae2c-8335-4c6c-b065-a8ce0c919f3b None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:14.323041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-24e1ae2c-8335-4c6c-b065-a8ce0c919f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:14.382038 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1442/2847] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:14.389977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92df1-69d5-4355-acee-df06bf007248 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.390265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92df1-69d5-4355-acee-df06bf007248 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.390544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92df1-69d5-4355-acee-df06bf007248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.413335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-36c92df1-69d5-4355-acee-df06bf007248 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:14.414539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-36c92df1-69d5-4355-acee-df06bf007248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:14.422450 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1406/2848] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:14.426624 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1ae2c-8335-4c6c-b065-a8ce0c919f3b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.436290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1ae2c-8335-4c6c-b065-a8ce0c919f3b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.440884 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.441642 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.441911 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.442183 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.442825 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:05:14.443184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.444786 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.456225 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1443/2849] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:05:14.460793 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92df1-69d5-4355-acee-df06bf007248 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.469856 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92df1-69d5-4355-acee-df06bf007248 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.475165 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.475757 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.476061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.476329 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.476949 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.478861 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.507612 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1407/2850] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:05:14 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:05:14.509954 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.520351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d186868-814c-4bb4-962b-79d8b5e42e33 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.575277 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.575867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.576145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.576415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.577045 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.578991 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.600399 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1444/2851] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:05:14 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:05:14.605230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.615345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf08fc-d566-4561-b448-47e6ebe08c87 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.619907 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.620496 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.620774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.621046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.621716 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.623413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.634383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1408/2852] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:05:14.638475 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.650155 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c12f6-c4c0-46d2-9d0e-e2dc91b8f721 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.655605 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c0290-94c5-4a11-93d6-48878b484253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.656209 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0290-94c5-4a11-93d6-48878b484253 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.656488 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0290-94c5-4a11-93d6-48878b484253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.657018 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0290-94c5-4a11-93d6-48878b484253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.657730 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c0290-94c5-4a11-93d6-48878b484253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.659328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c0290-94c5-4a11-93d6-48878b484253 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.669985 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1445/2853] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:05:14.674048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.683826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a582ba-3a6c-42e7-8bc9-7a06bd9909e1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.688459 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.689093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.689404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.689678 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] PATH_INFO: `/v3/projects/4935439b82154c548b0bfad252c407b7/users/fd32d1d55f274ac8ac7113a8f29349a0/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.704430 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4935439b82154c548b0bfad252c407b7, user_id=fd32d1d55f274ac8ac7113a8f29349a0, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.707211 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.720610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1409/2854] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:05:14 2022] PUT /identity/v3/projects/4935439b82154c548b0bfad252c407b7/users/fd32d1d55f274ac8ac7113a8f29349a0/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:05:14.730959 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cf230-efb6-4678-89bf-4248878dbe5b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.731246 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cf230-efb6-4678-89bf-4248878dbe5b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.731539 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cf230-efb6-4678-89bf-4248878dbe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.758778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c94cf230-efb6-4678-89bf-4248878dbe5b None None] MFA Rules not processed for user `fd32d1d55f274ac8ac7113a8f29349a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:14.759995 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c94cf230-efb6-4678-89bf-4248878dbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:14.813479 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:14.823462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1446/2855] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:14.834530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:14.836242 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:14.891930 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ec2b-27b7-4213-bd7f-473aa8decf1a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:14.896220 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:14.896818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:14.897107 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:14.897412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:14.905555 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:14.907218 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:14.915972 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1410/2856] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:05:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:05:16.268406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9732a-8b9e-4987-96b8-4d79c968a3f7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:16.268890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9732a-8b9e-4987-96b8-4d79c968a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:16.268890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9732a-8b9e-4987-96b8-4d79c968a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:16.288252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f5d9732a-8b9e-4987-96b8-4d79c968a3f7 None None] MFA Rules not processed for user `fd32d1d55f274ac8ac7113a8f29349a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:16.289579 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f5d9732a-8b9e-4987-96b8-4d79c968a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:16.296585 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1447/2857] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:21.005587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:21.007413 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:21.095068 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:21.140147 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e6778-cf3a-4cb7-9efe-a708a04b85a0 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:21.145843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:21.146568 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:21.146922 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:21.147197 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:21.148962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:21.162368 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:21.163957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:21.176253 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1411/2858] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:05:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:05:26.283495 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:31.170002 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=20 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:42.863641 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba29f6d-9c66-4495-a7ea-c14139663ae0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:42.864171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba29f6d-9c66-4495-a7ea-c14139663ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:42.864289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba29f6d-9c66-4495-a7ea-c14139663ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:42.893498 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4ba29f6d-9c66-4495-a7ea-c14139663ae0 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:42.895185 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4ba29f6d-9c66-4495-a7ea-c14139663ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:42.954041 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1448/2859] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:42.998649 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:43.000378 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:43.070707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958e80b-2493-4ca4-a6ee-35d70b727930 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:43.076834 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:43.077482 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:43.077844 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:43.078123 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:43.088050 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:43.089677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:43.097882 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1412/2860] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:05:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:05:43.610236 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0c78e-7710-41a8-9c1f-3903f3687c7f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:43.610519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0c78e-7710-41a8-9c1f-3903f3687c7f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:43.610795 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0c78e-7710-41a8-9c1f-3903f3687c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:43.631301 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-84e0c78e-7710-41a8-9c1f-3903f3687c7f None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:43.632512 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-84e0c78e-7710-41a8-9c1f-3903f3687c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:43.639953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1449/2861] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:05:43.651932 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:43.662609 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0133-c0f4-478e-94c3-2e74b2825b9b service swift] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:43.667559 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:43.668140 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:43.668419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:43.668720 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:43.677061 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:43.678664 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:43.686493 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1413/2862] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:05:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:05:43.911894 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8af35-dc57-4ad8-9b69-c57ab1f87b93 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:43.912343 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8af35-dc57-4ad8-9b69-c57ab1f87b93 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:43.912437 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8af35-dc57-4ad8-9b69-c57ab1f87b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:43.931948 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-94e8af35-dc57-4ad8-9b69-c57ab1f87b93 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:43.933126 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-94e8af35-dc57-4ad8-9b69-c57ab1f87b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:43.940135 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1450/2863] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:43.951938 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:43.962405 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73807b2-7ad9-4468-aea9-6a72f67c181e service swift] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:43.967501 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:43.968037 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:43.968313 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:43.968583 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:43.977099 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:43.978739 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:43.986471 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1414/2864] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:05:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:05:44.301794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71e31-deb7-45b7-928d-be13fb1e81f0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:44.302082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71e31-deb7-45b7-928d-be13fb1e81f0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:44.302359 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71e31-deb7-45b7-928d-be13fb1e81f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:44.321563 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fab71e31-deb7-45b7-928d-be13fb1e81f0 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:44.322748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fab71e31-deb7-45b7-928d-be13fb1e81f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:44.329798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1451/2865] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:44.342282 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:44.353192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbe2d9-234d-411e-bfd6-af5c2969b56e service swift] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:44.358136 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f878ad-d59a-4a7d-8f4d-c9b75b986a6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:44.358717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f878ad-d59a-4a7d-8f4d-c9b75b986a6f service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:44.358996 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f878ad-d59a-4a7d-8f4d-c9b75b986a6f service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:44.359265 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f878ad-d59a-4a7d-8f4d-c9b75b986a6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:44.366952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f878ad-d59a-4a7d-8f4d-c9b75b986a6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:44.368497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f878ad-d59a-4a7d-8f4d-c9b75b986a6f service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:44.376240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1415/2866] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:05:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:05:46.442962 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71e31-deb7-45b7-928d-be13fb1e81f0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:46.502438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71e31-deb7-45b7-928d-be13fb1e81f0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:46.507568 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:46.508176 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:46.508450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:46.508830 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] PATH_INFO: `/v3/users/fd32d1d55f274ac8ac7113a8f29349a0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:46.509970 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd32d1d55f274ac8ac7113a8f29349a0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:46.511869 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:46.555490 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd32d1d55f274ac8ac7113a8f29349a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:46.562274 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd32d1d55f274ac8ac7113a8f29349a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:46.573555 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:46.574907 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd32d1d55f274ac8ac7113a8f29349a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:46.575707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd32d1d55f274ac8ac7113a8f29349a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:46.583749 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f44ce98f-e35b-4160-b72b-c9f771a34ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd32d1d55f274ac8ac7113a8f29349a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:46.589598 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1452/2867] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:05:46 2022] DELETE /identity/v3/users/fd32d1d55f274ac8ac7113a8f29349a0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:05:46.593944 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:46.594441 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:46.594539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:46.597134 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:46.597647 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:46.598097 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:46.614526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:46.615729 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:46.618772 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] MFA Rules not processed for user `33a5368bd6fb4ebeba76c5f6ce0c7c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:46.619958 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:46.674301 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1416/2868] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:46.676424 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1453/2869] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:05:46.684271 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:46.685928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:46.742477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:46.766328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ead34-5748-4ee3-b40c-4e747a151a80 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:46.769954 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75453569-ae1d-4b82-8b36-63e12fb5595a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:46.770835 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453569-ae1d-4b82-8b36-63e12fb5595a service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:46.771114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453569-ae1d-4b82-8b36-63e12fb5595a service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:46.771383 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453569-ae1d-4b82-8b36-63e12fb5595a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:46.780666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75453569-ae1d-4b82-8b36-63e12fb5595a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:46.782365 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75453569-ae1d-4b82-8b36-63e12fb5595a service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:46.791652 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1417/2870] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:05:46.968996 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:46.970457 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:46.986701 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85e5c7-3fcf-463f-b4fa-2fd9f88396e3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.004064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.004728 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.005005 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.005308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] PATH_INFO: `/v3/projects/4935439b82154c548b0bfad252c407b7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.006480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4935439b82154c548b0bfad252c407b7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.008425 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.061980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4935439b82154c548b0bfad252c407b7', 'request_id': 'req-23520496-f44f-4987-9805-3cb88e0600a5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:47.068316 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4935439b82154c548b0bfad252c407b7', 'request_id': 'req-23520496-f44f-4987-9805-3cb88e0600a5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:47.073997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4935439b82154c548b0bfad252c407b7', 'request_id': 'req-23520496-f44f-4987-9805-3cb88e0600a5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:47.088368 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] The token cache is being invalidate because project 4935439b82154c548b0bfad252c407b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:05:47.088945 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:47.089760 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-23520496-f44f-4987-9805-3cb88e0600a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:05:47.091469 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1454/2871] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:05:46 2022] DELETE /identity/v3/projects/4935439b82154c548b0bfad252c407b7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:05:47.096399 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4402b8-6a6f-4c86-9327-71a9ee551c10 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.096717 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4402b8-6a6f-4c86-9327-71a9ee551c10 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.097544 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4402b8-6a6f-4c86-9327-71a9ee551c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.120166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ec4402b8-6a6f-4c86-9327-71a9ee551c10 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:47.121401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ec4402b8-6a6f-4c86-9327-71a9ee551c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:47.182865 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1418/2872] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:47.190713 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22ca66-8267-4ab1-92b3-5334d7c2f999 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.191004 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22ca66-8267-4ab1-92b3-5334d7c2f999 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.191304 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22ca66-8267-4ab1-92b3-5334d7c2f999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.211334 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cf22ca66-8267-4ab1-92b3-5334d7c2f999 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:47.212613 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cf22ca66-8267-4ab1-92b3-5334d7c2f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:47.220499 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1455/2873] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:47.225395 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4402b8-6a6f-4c86-9327-71a9ee551c10 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:47.234904 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4402b8-6a6f-4c86-9327-71a9ee551c10 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.239748 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.240353 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.240630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.240901 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.241934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:05:47.241934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.243442 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.258818 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1419/2874] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:05:47.260078 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22ca66-8267-4ab1-92b3-5334d7c2f999 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:47.270099 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22ca66-8267-4ab1-92b3-5334d7c2f999 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.274935 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.275541 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.275856 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.276130 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.276758 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.278740 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.310152 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1456/2875] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:05:47 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:05:47.311326 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:47.321494 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbaa99-b196-492a-ab9e-93f5305e2b68 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.378783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.379393 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.379668 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.379940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.380572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.382526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.410998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1420/2876] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:05:47 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:05:47.412149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:47.422230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23808b20-a429-4829-8299-c60102005b8c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.427638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.428206 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.428480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.428747 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.429416 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.430998 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.443001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1457/2877] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:05:47.447419 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:47.457830 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fcf7f-fb20-4f7a-8436-96bdf14e85cd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.462412 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c28a5b-5952-471d-b8de-215de7baa713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.463066 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c28a5b-5952-471d-b8de-215de7baa713 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.463289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c28a5b-5952-471d-b8de-215de7baa713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.463558 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c28a5b-5952-471d-b8de-215de7baa713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.464184 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c28a5b-5952-471d-b8de-215de7baa713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.465848 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c28a5b-5952-471d-b8de-215de7baa713 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.479287 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1421/2878] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 15:05:47.483503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:47.493588 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b30da-92c7-4544-b6d3-6b98f67f857b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.498454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.499226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.499506 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.499798 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] PATH_INFO: `/v3/projects/2af23e9975024ada9e14dba49669e119/users/28119e034d764c10b8736f7b63f96d9a/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.516739 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af23e9975024ada9e14dba49669e119, user_id=28119e034d764c10b8736f7b63f96d9a, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.521491 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.549459 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1458/2879] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:05:47 2022] PUT /identity/v3/projects/2af23e9975024ada9e14dba49669e119/users/28119e034d764c10b8736f7b63f96d9a/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:05:47.608530 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aa4e8-9584-4391-a877-62153a5aa459 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.609127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aa4e8-9584-4391-a877-62153a5aa459 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.609127 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aa4e8-9584-4391-a877-62153a5aa459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.640552 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b66aa4e8-9584-4391-a877-62153a5aa459 None None] MFA Rules not processed for user `28119e034d764c10b8736f7b63f96d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:05:47.641811 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b66aa4e8-9584-4391-a877-62153a5aa459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:47.671905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:47.711464 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1422/2880] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:05:47.722556 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:05:47.724503 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:05:47.743050 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:47.809788 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3226b6-163b-47f1-8bd9-b2b35cbd140c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:05:47.813654 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3caf65-aecd-4732-b1bf-fe178a85ef8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:05:47.814262 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3caf65-aecd-4732-b1bf-fe178a85ef8a service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:05:47.814543 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3caf65-aecd-4732-b1bf-fe178a85ef8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:05:47.814819 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3caf65-aecd-4732-b1bf-fe178a85ef8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:05:47.824694 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3caf65-aecd-4732-b1bf-fe178a85ef8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:05:47.826707 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3caf65-aecd-4732-b1bf-fe178a85ef8a service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:05:47.837901 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1459/2881] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:05:57.707708 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=14 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:05:57.833145 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=29 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:04.134262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aa4e8-9584-4391-a877-62153a5aa459 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:04.198048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aa4e8-9584-4391-a877-62153a5aa459 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:04.202508 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:04.203133 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:04.203404 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:04.203674 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] PATH_INFO: `/v3/users/28119e034d764c10b8736f7b63f96d9a` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:04.204788 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=28119e034d764c10b8736f7b63f96d9a)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:04.207264 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:04.260560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28119e034d764c10b8736f7b63f96d9a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.272962 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28119e034d764c10b8736f7b63f96d9a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.279662 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28119e034d764c10b8736f7b63f96d9a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.287672 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28119e034d764c10b8736f7b63f96d9a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.288357 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28119e034d764c10b8736f7b63f96d9a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.294358 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1423/2882] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:06:04 2022] DELETE /identity/v3/users/28119e034d764c10b8736f7b63f96d9a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:06:04.298789 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:04.299243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:04.299362 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:04.318260 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:04.319447 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:04.375404 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1460/2883] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:04.385489 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:04.386933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:04.443337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abf897-a5ec-4212-9022-8c872839ec4b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:04.447031 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d550a2-e803-4e09-9be7-edb7028c6d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:04.447630 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d550a2-e803-4e09-9be7-edb7028c6d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:04.447902 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d550a2-e803-4e09-9be7-edb7028c6d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:04.448176 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d550a2-e803-4e09-9be7-edb7028c6d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:04.456912 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d550a2-e803-4e09-9be7-edb7028c6d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:04.458345 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d550a2-e803-4e09-9be7-edb7028c6d22 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:04.467470 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1424/2884] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:06:04.726530 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:04.727949 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:04.748826 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cbaf-7f5a-4080-9255-dc1a2da8079e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:04.754210 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:04.754847 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:04.755125 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:04.755397 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] PATH_INFO: `/v3/projects/2af23e9975024ada9e14dba49669e119` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:04.756513 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af23e9975024ada9e14dba49669e119)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:04.758484 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:04.803778 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af23e9975024ada9e14dba49669e119', 'request_id': 'req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.810012 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af23e9975024ada9e14dba49669e119', 'request_id': 'req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.815988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af23e9975024ada9e14dba49669e119', 'request_id': 'req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.827968 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] The token cache is being invalidate because project 2af23e9975024ada9e14dba49669e119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:06:04.828257 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.828868 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8bda6ebe-d215-4b17-85fd-e92a4f9deb8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:06:04.830350 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1461/2885] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:06:04 2022] DELETE /identity/v3/projects/2af23e9975024ada9e14dba49669e119 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:06:04.835335 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9ff2-ea3b-4622-9423-bb8177435693 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:04.835622 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9ff2-ea3b-4622-9423-bb8177435693 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:04.835896 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9ff2-ea3b-4622-9423-bb8177435693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:04.856416 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-fc4d9ff2-ea3b-4622-9423-bb8177435693 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:04.857634 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fc4d9ff2-ea3b-4622-9423-bb8177435693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:04.874549 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:04.917923 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1425/2886] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:04.926115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528559f-001e-415b-9384-516869a792e7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:04.926405 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528559f-001e-415b-9384-516869a792e7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:04.926679 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528559f-001e-415b-9384-516869a792e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:04.946019 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e528559f-001e-415b-9384-516869a792e7 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:04.947233 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e528559f-001e-415b-9384-516869a792e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:04.954990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1462/2887] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:04.959046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9ff2-ea3b-4622-9423-bb8177435693 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:04.968328 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9ff2-ea3b-4622-9423-bb8177435693 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:04.973337 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:04.973918 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:04.974194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:04.974462 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:04.975094 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 15:06:04.975388 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:04.976946 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:04.988590 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1426/2888] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 15:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 15:06:04.992646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528559f-001e-415b-9384-516869a792e7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:05.001801 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528559f-001e-415b-9384-516869a792e7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:05.006507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:05.007090 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.007391 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.007661 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.008289 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:05.010281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:05.040450 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1463/2889] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:06:04 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:06:05.044998 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:05.055497 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a18e8-21d8-4733-b62c-1e2aefa5b2eb admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:05.108955 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:05.109505 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.109783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.110052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.110679 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:05.113183 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:05.134341 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1427/2890] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:06:05 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 15:06:05.138400 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:05.148861 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574063f-73c4-4dae-8261-667103d3606f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:05.153480 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:05.154068 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.154345 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.154615 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.155252 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:05.156828 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:05.167228 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1464/2891] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:06:05.171145 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:05.181843 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378093cb-ef17-439c-9686-c32a7cf72d98 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:05.186513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee73bab-2b5e-4675-8fcf-66c2ea54e7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:05.187097 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee73bab-2b5e-4675-8fcf-66c2ea54e7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.187379 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee73bab-2b5e-4675-8fcf-66c2ea54e7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.187656 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee73bab-2b5e-4675-8fcf-66c2ea54e7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.188300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee73bab-2b5e-4675-8fcf-66c2ea54e7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:05.189919 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee73bab-2b5e-4675-8fcf-66c2ea54e7f0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:05.200314 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1428/2892] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:06:05.204231 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:05.214516 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95f2ac-b2d5-4816-9e4b-55fb99c034f1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:05.219102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:05.219729 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.220036 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.220308 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] PATH_INFO: `/v3/projects/ee3a90f56942438a887a45157622532f/users/a0c64b4ade2e4fdda31999e89e307980/roles/72e8b68a7d364c4bb0fcf9e444949d58` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.234964 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3a90f56942438a887a45157622532f, user_id=a0c64b4ade2e4fdda31999e89e307980, role_id=72e8b68a7d364c4bb0fcf9e444949d58)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:05.237698 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:05.254462 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1465/2893] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:06:05 2022] PUT /identity/v3/projects/ee3a90f56942438a887a45157622532f/users/a0c64b4ade2e4fdda31999e89e307980/roles/72e8b68a7d364c4bb0fcf9e444949d58 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:06:05.256392 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.257277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.257277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.276303 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] MFA Rules not processed for user `3ec2e03066934f2597854e78a5c4d345`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:05.277587 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:05.337336 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1429/2894] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 15:06:05.348291 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:05.350116 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:05.371848 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:05.407955 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b9f7-3db3-42f1-b495-8494edbcd9e7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:05.412390 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08144b20-cb28-4f61-8385-2a8dfe47618a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:05.412980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08144b20-cb28-4f61-8385-2a8dfe47618a service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.413293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08144b20-cb28-4f61-8385-2a8dfe47618a service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.413562 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08144b20-cb28-4f61-8385-2a8dfe47618a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.422357 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08144b20-cb28-4f61-8385-2a8dfe47618a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:05.423777 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08144b20-cb28-4f61-8385-2a8dfe47618a service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:05.432990 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1466/2895] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:06:05.670498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:05.672007 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:05.704240 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:05.733539 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f86ef0-c415-4446-a3de-7bb5a1c6b4a1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:05.738008 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46482f26051455daeed2c00f8ced4bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:05.738773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:05.738773 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:05.739084 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:05.740815 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:05.809448 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:05.811981 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:05.822387 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1430/2896] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:06:09.670665 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20adce-d1d5-4893-a56d-7546bb73cdf6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:09.671169 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20adce-d1d5-4893-a56d-7546bb73cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:09.671293 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20adce-d1d5-4893-a56d-7546bb73cdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:09.701270 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8a20adce-d1d5-4893-a56d-7546bb73cdf6 None None] MFA Rules not processed for user `a0c64b4ade2e4fdda31999e89e307980`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:09.702486 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8a20adce-d1d5-4893-a56d-7546bb73cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:09.769411 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1467/2897] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:09.774012 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:09.786036 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e2355-603a-4704-b53b-263745e93250 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:09.791011 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:09.791627 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:09.791978 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:09.792319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:09.793063 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:09.795212 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:09.824465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1431/2898] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 15:06:09 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:06:09.828931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20adce-d1d5-4893-a56d-7546bb73cdf6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:09.840281 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20adce-d1d5-4893-a56d-7546bb73cdf6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:09.912121 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:09.912638 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:09.912916 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:09.913184 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:09.913857 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:09.915999 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:09.940082 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1468/2899] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 15:06:09 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 15:06:09.944868 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:09.958799 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c191-5042-43bd-8f8c-39c60ecaa1a8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:09.965569 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:09.966175 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:09.966458 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:09.966734 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:09.967384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:09.968973 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:09.980047 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1432/2900] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:06:09.984215 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:09.998947 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67487732-1400-4ece-85dd-32f39774bdd8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:10.004415 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:10.005064 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:10.005377 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:10.005652 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] PATH_INFO: `/v3/projects/6030c05db1e04aa1a3554b6021b8b558/users/2ef4187d65cb48a8ba7e43e08262f265/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:10.020732 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6030c05db1e04aa1a3554b6021b8b558, user_id=2ef4187d65cb48a8ba7e43e08262f265, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:10.023412 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:10.037051 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1469/2901] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:06:09 2022] PUT /identity/v3/projects/6030c05db1e04aa1a3554b6021b8b558/users/2ef4187d65cb48a8ba7e43e08262f265/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:06:10.041076 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:10.051560 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa67fb0-4031-4319-9e15-2418fef82374 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:10.111506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d9715-b0c5-424f-af48-d56fd28975b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:10.112701 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d9715-b0c5-424f-af48-d56fd28975b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:10.112977 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d9715-b0c5-424f-af48-d56fd28975b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:10.113277 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d9715-b0c5-424f-af48-d56fd28975b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:10.113915 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d9715-b0c5-424f-af48-d56fd28975b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:10.115485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d9715-b0c5-424f-af48-d56fd28975b9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:10.126709 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1433/2902] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 15:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 15:06:10.130845 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:10.142279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fec28-f6b9-4f52-9799-0a5de2e79f40 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:10.147267 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:10.148027 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:10.148373 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:10.148714 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] PATH_INFO: `/v3/projects/6030c05db1e04aa1a3554b6021b8b558/users/2ef4187d65cb48a8ba7e43e08262f265/roles/82245735988747a4bc5704727a1cf374` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:10.150794 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6030c05db1e04aa1a3554b6021b8b558, user_id=2ef4187d65cb48a8ba7e43e08262f265, role_id=82245735988747a4bc5704727a1cf374)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:10.153675 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:10.165493 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1470/2903] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 15:06:10 2022] PUT /identity/v3/projects/6030c05db1e04aa1a3554b6021b8b558/users/2ef4187d65cb48a8ba7e43e08262f265/roles/82245735988747a4bc5704727a1cf374 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 15:06:14.830299 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c88fee-0080-4108-a3df-c4f35beebf7a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:14.831610 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c88fee-0080-4108-a3df-c4f35beebf7a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:14.832126 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c88fee-0080-4108-a3df-c4f35beebf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:14.884005 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-14c88fee-0080-4108-a3df-c4f35beebf7a None None] MFA Rules not processed for user `2ef4187d65cb48a8ba7e43e08262f265`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:14.885400 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-14c88fee-0080-4108-a3df-c4f35beebf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:14.923629 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:14.969043 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1434/2904] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:14.979507 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:14.981438 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:14.989328 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:15.058687 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c99f-974d-4869-823f-4721679f084e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:15.062535 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d1fc63b356499d80061e358fb447e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:15.063149 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:15.063429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:15.063704 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:15.122561 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:15.124021 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:15.133584 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1471/2905] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:06:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 15:06:15.265639 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccff86-b5e9-4aca-86e0-933372395c94 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:15.265934 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccff86-b5e9-4aca-86e0-933372395c94 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:15.266209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccff86-b5e9-4aca-86e0-933372395c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:15.286854 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-44ccff86-b5e9-4aca-86e0-933372395c94 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:15.288057 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-44ccff86-b5e9-4aca-86e0-933372395c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:15.348149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1435/2906] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:15.360044 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:15.361444 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:15.428966 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5208bc-10b6-40f5-8ab4-68091f91fc7a service glance] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:15.434102 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ae717-183f-42f8-a052-2a51b57ceda1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:15.434700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ae717-183f-42f8-a052-2a51b57ceda1 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:15.434975 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ae717-183f-42f8-a052-2a51b57ceda1 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:15.435243 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ae717-183f-42f8-a052-2a51b57ceda1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:15.444166 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ae717-183f-42f8-a052-2a51b57ceda1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:15.445779 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ae717-183f-42f8-a052-2a51b57ceda1 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:15.453967 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1472/2907] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:06:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:06:17.088101 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bcfc-42cc-4e90-8696-2aee213f413f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:17.088645 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bcfc-42cc-4e90-8696-2aee213f413f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:17.088765 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bcfc-42cc-4e90-8696-2aee213f413f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:17.115485 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bb94bcfc-42cc-4e90-8696-2aee213f413f None None] MFA Rules not processed for user `9a50fbbf18584bb1a784283e768be7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:17.118401 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bb94bcfc-42cc-4e90-8696-2aee213f413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:17.207528 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1436/2908] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:25.450587 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=78 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:26.358946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77558c09-7e9d-4cc9-a135-e79a2fddc2c0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:26.359230 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77558c09-7e9d-4cc9-a135-e79a2fddc2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:26.359500 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77558c09-7e9d-4cc9-a135-e79a2fddc2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:26.378247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-77558c09-7e9d-4cc9-a135-e79a2fddc2c0 None None] MFA Rules not processed for user `a0c64b4ade2e4fdda31999e89e307980`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:26.379439 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-77558c09-7e9d-4cc9-a135-e79a2fddc2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:26.386456 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1473/2909] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:27.182928 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=63 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:29.146885 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:29.147486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:29.147486 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:29.170826 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:06:29.172202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:29.188722 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1437/2910] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:06:29.202600 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77558c09-7e9d-4cc9-a135-e79a2fddc2c0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:29.214872 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77558c09-7e9d-4cc9-a135-e79a2fddc2c0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:29.219858 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:29.220558 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:29.221414 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:29.221691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:29.230187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:29.231791 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:29.241171 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1474/2911] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:06:39.164935 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:39.237279 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=9 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:06:45.567048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:06:45.568440 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:45.624511 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147f179-98dc-45be-a4e6-ff2775fa0964 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:06:45.628069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:06:45.628666 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:06:45.628940 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:06:45.629217 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:06:45.630933 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:06:45.695966 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:06:45.697423 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:06:45.707695 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1438/2912] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:06:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:06:55.702589 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=50 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:07:00.994953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:07:01.004245 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857caa40-fdfe-462f-925d-8160bec21fc8 service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:07:01.008261 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:07:01.008860 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:07:01.009115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:07:01.009406 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:07:01.018393 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:07:01.019807 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:07:01.029769 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1475/2913] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:07:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:07:03.668074 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:07:03.669819 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:07:03.779608 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73b2b9-6bc9-4340-aff1-0415b7c95de9 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:07:03.786054 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:07:03.788048 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:07:03.788381 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:07:03.788705 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] PATH_INFO: `/v3/users/33a5368bd6fb4ebeba76c5f6ce0c7c28` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:07:03.790291 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a5368bd6fb4ebeba76c5f6ce0c7c28)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:07:03.793193 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:07:03.856069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a5368bd6fb4ebeba76c5f6ce0c7c28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:03.871039 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a5368bd6fb4ebeba76c5f6ce0c7c28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:03.881499 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a5368bd6fb4ebeba76c5f6ce0c7c28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:03.890874 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a5368bd6fb4ebeba76c5f6ce0c7c28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:03.891950 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a5368bd6fb4ebeba76c5f6ce0c7c28'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:03.904102 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1439/2914] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:07:03 2022] DELETE /identity/v3/users/33a5368bd6fb4ebeba76c5f6ce0c7c28 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:07:04.138319 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:07:04.156285 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6a7e3-5fd5-4c03-ba5b-c1264a5a0fc0 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:07:04.234870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:07:04.235527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:07:04.236165 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:07:04.236614 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] PATH_INFO: `/v3/projects/781a403f041b4389b804107c30ffedac` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:07:04.238076 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=781a403f041b4389b804107c30ffedac)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:07:04.240150 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:07:04.299997 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781a403f041b4389b804107c30ffedac', 'request_id': 'req-14c8706b-a8eb-426e-8138-7c6eea67a77f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:04.306726 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781a403f041b4389b804107c30ffedac', 'request_id': 'req-14c8706b-a8eb-426e-8138-7c6eea67a77f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:04.313521 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781a403f041b4389b804107c30ffedac', 'request_id': 'req-14c8706b-a8eb-426e-8138-7c6eea67a77f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:04.335890 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] The token cache is being invalidate because project 781a403f041b4389b804107c30ffedac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:07:04.336264 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:04.337001 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:07:04.338699 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1476/2915] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:07:04 2022] DELETE /identity/v3/projects/781a403f041b4389b804107c30ffedac => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:07:13.895192 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:07:14.331234 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:07:21.263750 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:07:21.265398 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:07:21.322805 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13364c-1013-4286-b934-8b6f2d74213f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:07:21.372952 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:07:21.373603 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:07:21.373887 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:07:21.374166 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:07:21.375905 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:07:21.461052 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:07:21.462526 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:07:21.483085 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1440/2916] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:07:22.776157 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:07:22.777582 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:07:22.790349 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8706b-a8eb-426e-8138-7c6eea67a77f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:07:22.794011 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:07:22.794597 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:07:22.794874 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:07:22.795140 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:07:22.796802 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:07:22.810458 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:07:22.811882 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:07:22.822700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1477/2917] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:07:31.480431 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=83 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:07:32.817833 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:06.554703 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:06.556878 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:06.575149 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93217e5a-58e0-4401-8581-d6ea0ccd4f27 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:06.581535 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:06.582205 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:06.582490 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:06.582760 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:06.585159 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:06.600477 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:06.602877 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:06.615831 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1441/2918] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:08:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:08:06.684748 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:06.695142 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156a96-192d-4d19-be78-370049570c7b service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:06.699335 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3be2ac-686b-4a40-abec-3c2c7100f06f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:06.700186 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3be2ac-686b-4a40-abec-3c2c7100f06f service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:06.700671 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3be2ac-686b-4a40-abec-3c2c7100f06f service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:06.701326 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3be2ac-686b-4a40-abec-3c2c7100f06f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:06.703272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3f3be2ac-686b-4a40-abec-3c2c7100f06f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:06.714618 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3be2ac-686b-4a40-abec-3c2c7100f06f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:06.716151 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3be2ac-686b-4a40-abec-3c2c7100f06f service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:06.726272 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1478/2919] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:08:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:08:07.596465 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:07.608289 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e53d4-bfe9-435f-b49f-1baf6b84305b service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:07.615230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b6809bf91d244e4bed5b8a256a3f95a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:07.616114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:07.616435 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:07.617338 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:07.619613 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:07.633653 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:07.635567 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:07.648753 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1442/2920] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:08:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:08:13.972083 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:13.972611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:13.972611 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:14.027908 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] MFA Rules not processed for user `b3e63489796544afaa44deab302681b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 15:08:14.029127 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:14.091854 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1479/2921] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 15:08:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 15:08:14.133230 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:14.134635 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:14.224541 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8387790-9d11-4fd0-b727-acbc7995e1ec service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:14.229437 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '552b3ea5ceab4114908b8108d415c9cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:14.230044 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:14.230320 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:14.230591 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:14.238382 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:14.239956 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:14.247847 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1443/2922] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 15:08:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 15:08:21.418244 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:21.419691 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:21.485519 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4af04-5ff5-45ba-8ffa-5541d4a80b03 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:21.490043 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:21.490670 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:21.490953 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:21.491226 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] PATH_INFO: `/v3/users/a0c64b4ade2e4fdda31999e89e307980` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:21.492351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0c64b4ade2e4fdda31999e89e307980)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:21.494321 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:21.539834 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0c64b4ade2e4fdda31999e89e307980'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.546677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0c64b4ade2e4fdda31999e89e307980'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.551580 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:21.560201 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0c64b4ade2e4fdda31999e89e307980'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.560870 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0c64b4ade2e4fdda31999e89e307980'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.568646 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0c64b4ade2e4fdda31999e89e307980'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.574988 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1480/2923] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:08:21 2022] DELETE /identity/v3/users/a0c64b4ade2e4fdda31999e89e307980 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:21.748772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:21.760707 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ee3d3-fba8-4a2f-9121-15d5d78aa246 service swift] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:21.820774 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:21.821434 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:21.821829 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:21.822103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] PATH_INFO: `/v3/projects/ee3a90f56942438a887a45157622532f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:21.823202 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3a90f56942438a887a45157622532f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:21.825148 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:21.867173 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:21.870306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3a90f56942438a887a45157622532f', 'request_id': 'req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.876957 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3a90f56942438a887a45157622532f', 'request_id': 'req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.883093 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3a90f56942438a887a45157622532f', 'request_id': 'req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.894810 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] The token cache is being invalidate because project ee3a90f56942438a887a45157622532f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:08:21.895142 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.895783 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:21.897319 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1444/2924] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:08:21 2022] DELETE /identity/v3/projects/ee3a90f56942438a887a45157622532f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:21.901342 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:21.902875 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:21.916504 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7067c-b25c-4f4b-af32-a74913f9ca5b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:21.978745 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:21.979420 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:21.979700 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:21.979976 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] PATH_INFO: `/v3/users/2ef4187d65cb48a8ba7e43e08262f265` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:21.981115 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef4187d65cb48a8ba7e43e08262f265)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:21.983344 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:22.032454 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef4187d65cb48a8ba7e43e08262f265'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.040147 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef4187d65cb48a8ba7e43e08262f265'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.052327 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef4187d65cb48a8ba7e43e08262f265'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.052982 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef4187d65cb48a8ba7e43e08262f265'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.060809 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef4187d65cb48a8ba7e43e08262f265'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.067053 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1481/2925] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:08:21 2022] DELETE /identity/v3/users/2ef4187d65cb48a8ba7e43e08262f265 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:22.244300 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:22.246046 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:22.258953 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a543f9b-68f0-4b63-be43-14a6db7e6d3b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:22.317322 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:22.317963 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:22.318241 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:22.318513 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] PATH_INFO: `/v3/projects/6030c05db1e04aa1a3554b6021b8b558` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:22.319624 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6030c05db1e04aa1a3554b6021b8b558)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:22.321576 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:22.368262 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6030c05db1e04aa1a3554b6021b8b558', 'request_id': 'req-cc5fa82a-d569-4bef-b4c5-a1b1db529662'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.374384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6030c05db1e04aa1a3554b6021b8b558', 'request_id': 'req-cc5fa82a-d569-4bef-b4c5-a1b1db529662'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.380088 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6030c05db1e04aa1a3554b6021b8b558', 'request_id': 'req-cc5fa82a-d569-4bef-b4c5-a1b1db529662'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.392921 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] The token cache is being invalidate because project 6030c05db1e04aa1a3554b6021b8b558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:08:22.393279 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.393917 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:22.395994 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1445/2926] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:08:22 2022] DELETE /identity/v3/projects/6030c05db1e04aa1a3554b6021b8b558 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:29.179931 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:29.181429 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:29.238630 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58101d-9fce-428c-a646-859322ea0e48 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:29.247227 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:29.287781 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9edd8cf448e42fd8e888c8f531fa70d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:29.288380 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:29.288658 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:29.288926 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:29.290643 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:29.350677 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:29.352122 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:29.364371 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1482/2927] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:08:32.373929 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:32.375364 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:32.459487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fa82a-d569-4bef-b4c5-a1b1db529662 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:32.464488 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:32.465103 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:32.465403 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:32.465677 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:32.467506 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:32.477035 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:32.481498 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:32.483086 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:32.495572 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1446/2928] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:08:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:08:35.553187 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:35.554697 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:35.570329 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cca3a1-ab5a-48a0-bfa5-3a85d823a998 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:35.575639 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:35.576290 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:35.576570 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:35.576843 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] PATH_INFO: `/v3/users/9a50fbbf18584bb1a784283e768be7a2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:35.578030 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a50fbbf18584bb1a784283e768be7a2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:35.580275 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:35.626867 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a50fbbf18584bb1a784283e768be7a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.634241 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a50fbbf18584bb1a784283e768be7a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.647091 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a50fbbf18584bb1a784283e768be7a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.647767 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a50fbbf18584bb1a784283e768be7a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.654803 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a50fbbf18584bb1a784283e768be7a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.661131 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1483/2929] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:08:35 2022] DELETE /identity/v3/users/9a50fbbf18584bb1a784283e768be7a2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:35.830632 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:35.845681 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb71714-0ff7-4cd5-93fa-248b083ed774 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:35.902394 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:35.903026 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:35.903306 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:35.903581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] PATH_INFO: `/v3/projects/515d1ca3dcd249ada56b21a0a6e3a5b9` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:35.904694 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=515d1ca3dcd249ada56b21a0a6e3a5b9)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:35.906614 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:35.952514 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515d1ca3dcd249ada56b21a0a6e3a5b9', 'request_id': 'req-8f6d7079-c876-4ba4-9600-f385936346f7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.958562 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515d1ca3dcd249ada56b21a0a6e3a5b9', 'request_id': 'req-8f6d7079-c876-4ba4-9600-f385936346f7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.963948 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515d1ca3dcd249ada56b21a0a6e3a5b9', 'request_id': 'req-8f6d7079-c876-4ba4-9600-f385936346f7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.975598 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] The token cache is being invalidate because project 515d1ca3dcd249ada56b21a0a6e3a5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:08:35.975881 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.976487 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:35.978114 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1447/2930] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:08:35 2022] DELETE /identity/v3/projects/515d1ca3dcd249ada56b21a0a6e3a5b9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:35.982118 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:35.983527 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:35.997401 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03053d84-dee2-4bab-8870-7a03a457f33f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:36.031629 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:36.058672 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:36.059297 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:36.059578 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:36.059851 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] PATH_INFO: `/v3/users/6e0bcdb2603849cdbed230b141b7196e` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:36.060974 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0bcdb2603849cdbed230b141b7196e)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:36.062946 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:36.108595 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0bcdb2603849cdbed230b141b7196e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.115683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0bcdb2603849cdbed230b141b7196e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.127033 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0bcdb2603849cdbed230b141b7196e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.127688 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0bcdb2603849cdbed230b141b7196e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.136347 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0bcdb2603849cdbed230b141b7196e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.142247 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1484/2931] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 15:08:35 2022] DELETE /identity/v3/users/6e0bcdb2603849cdbed230b141b7196e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:36.315626 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:08:36.317110 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:08:36.334194 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d7079-c876-4ba4-9600-f385936346f7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:08:36.394295 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec2e03066934f2597854e78a5c4d345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aff011fa0544761b3d08887135e99aa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:08:36.394931 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:08:36.395209 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:08:36.395481 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] PATH_INFO: `/v3/projects/a8c71efcec054d2f8ae4ced89127d243` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:08:36.396581 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c71efcec054d2f8ae4ced89127d243)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:08:36.398515 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:08:36.429013 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:36.446744 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c71efcec054d2f8ae4ced89127d243', 'request_id': 'req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.452772 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c71efcec054d2f8ae4ced89127d243', 'request_id': 'req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.458342 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c71efcec054d2f8ae4ced89127d243', 'request_id': 'req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.470236 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] The token cache is being invalidate because project a8c71efcec054d2f8ae4ced89127d243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 15:08:36.470522 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.471128 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7d1fa0c7-cb62-458a-b65a-ecdfb7738052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 15:08:36.473069 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1448/2932] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 15:08:36 2022] DELETE /identity/v3/projects/a8c71efcec054d2f8ae4ced89127d243 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 15:08:46.138757 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:08:46.467384 dv-cinder-828696-11-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 15:10:23.666351 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 15:10:23.667752 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:10:23.735288 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2851-e3e6-4df9-9bc5-a7f86754dde2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 15:10:23.794077 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c7e6a-4ab7-4f53-8153-4e1ab347b3ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57cca115ce044ff5aacd5ea1a8834402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eb92812b7a74c53b321dd9e80955569', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 15:10:23.794683 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c7e6a-4ab7-4f53-8153-4e1ab347b3ed service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 15:10:23.794963 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c7e6a-4ab7-4f53-8153-4e1ab347b3ed service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 15:10:23.795240 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c7e6a-4ab7-4f53-8153-4e1ab347b3ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 15:10:23.796980 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ea5c7e6a-4ab7-4f53-8153-4e1ab347b3ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 15:10:23.808508 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c7e6a-4ab7-4f53-8153-4e1ab347b3ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 15:10:23.810120 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c7e6a-4ab7-4f53-8153-4e1ab347b3ed service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 15:10:23.820549 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1485/2933] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 15:10:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 15:10:33.815676 dv-cinder-828696-11-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=58 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}